SlideShare ist ein Scribd-Unternehmen logo
1 von 31
Downloaden Sie, um offline zu lesen
2011-10-25


Linux on System z
Optimizing Resource Utilization for Linux
under z/VM - Part1

                                                                    Dr. Juergen Doelle
                                                  IBM Germany Research & Development




      visit us at http://www.ibm.com/developerworks/linux/linux390/perf/index.html

                                                                                © 2011 IBM Corporation
Trademarks


IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International Business
Machines Corp., registered in many jurisdictions worldwide. Other product and service names
might be trademarks of IBM or other companies. A current list of IBM trademarks is available on
the Web at “Copyright and trademark information” at www.ibm.com/legal/copytrade.shtml.


Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both.


Java and all Java-based trademarks and logos are trademarks of Sun Microsystems, Inc. in the
United States, other countries, or both.


Other product and service names might be trademarks of IBM or other companies.




2                                                                                         © 2011 IBM Corporation
Agenda

    Introduction


    z/VM   Resource Manager (SRM)

    Recommendations    for Memory Overcommitment on z/VM

    Large   DCSS

    Summary




3                                                           © 2011 IBM Corporation
Introduction

     Running   virtualized guests provides significant advantages in regard of
       – Systems management
       – Resource utilization

     Running
            many systems with shared resources introduces a new dynamic into system
     compared to one system on one box
     Important   questions
       – Has the right system the right resources at the right moment?
       – Can I bring more guests online or increase guest sizes?
       – When does overcommitment become counterproductive?

     When   are the service level goals not longer met?
       – Which load situation is acceptable?

     Objectives   of that presentation
       – What can be done to optimize resource consumption, what are the options
       – Impact on WebSphere Application Sever environments and databases
       – Focus is on memory utilization




4                                                                                  © 2011 IBM Corporation
Agenda

    Introduction


    z/VM   Resource Manager (SRM)

    Recommendations    for Memory Overcommitment on z/VM

    Large   DCSS

    Summary




5                                                           © 2011 IBM Corporation
SRM Considerations for Linux guests

     The   responsibility of the SRM is:
       – to ensure that guests have the required resources available when scheduled
       – Guests are divided into queues Q1 – Q3 according to their interactivity
            • CMS guests are typically Q1 guests
            • Linux guests are typically Q3 guests

       – When a guest should be scheduled and SRM comes to the decision that the intended amount of resources can
         not be granted:
         ==> the guest is placed on the eligible list and waits

     SRM   settings might be overruled by enabling QUICKDISPATCH for the guests
       – But this should be used carefully, e.g. for business critical guests
       – Do not use QUICKDISPATCH for the majority of the guests, this limits z/VM's possibilities to dynamically
         manage the resource usage

     Consider   that Linux guests are running mostly in Q3!




6                                                                                                          © 2011 IBM Corporation
SRM Considerations for Linux guests

     Default   SRM settings (q srm)
       – LDUBUF : Q1=100% Q2=75% Q3=60%
       – STORBUF: Q1=125% Q2=105% Q3=95%

     LDUBUF     – to partition the system's paging resources
       – 60% for Q3 means:
         All Q3 guests together must use at maximum 60% of the paging resources
         if already used → eligible list
       – Recommendation:
         SET SRM LDUBUF 100 100 100
         to allow all Q3 guests to allocate the whole paging space

     STORBUF     – to partition host storage (central storage)
       – 95% for Q3 means:
         All Q3 guests together must use only 95% of the system storage
         → This prevents memory overcommitment when running Linux guests
       – Recommendation:
         SET SRM storbuf 300 250 200
         to allow all Q3 guests to allocate twice the amount of real storage
       – Depending on the level of overcommitment and amount of active/inactive guests, it might be necessary to go
         even higher, e.g. SET SRM storbuf 300 300 300

       – Ensure to have sufficient paging space!




7                                                                                                        © 2011 IBM Corporation
Agenda

    Introduction


    z/VM   Resource Manager (SRM)

    Recommendations    for Memory Overcommitment on z/VM

    Large   DCSS

    Summary




8                                                           © 2011 IBM Corporation
Memory overcommitment

     Memory     overcommitment is often mentioned as a major benefit of virtualized environments

     Memory overcommitment is the ability to use more virtual memory as physically available
     memory. It is based on the assumption that not all guests use their memory at the same time
     and/or some guests are over-sized in their memory setup.

     Recommendations     / limits are missing
     Very different definitions exist for the level of memory overcommitment
       – they are independent of the used middle ware
       – “Common” System z ratio is 3:1 virtual (guest memory) to physical memory
         E.g. run guests defined with a total of 3GB on 1 GB real memory
       – Performance can be heavily degraded when the memory overcommitment level is to high

     Goal:   Proved memory – overcommitment recommendations
     Identify   “rules”
       – Determine the minimum amount of physical memory required to run with an acceptable performance
       – Identify a dependency on the used middle ware / applications




9                                                                                                    © 2011 IBM Corporation
Tested Scenario


  Test       environment
        – Running a mix of server types as Linux guests on z/VM:
          LPAR with 28GB central storage + 2 GB expanded storage



          Guest workload                                 Guest Memory

          WebSphere Application Server                   13.5 GB (Java heaps 8GB)

          Database DB2                                   12.0 GB (memory pools about 2 GB)

          Tivoli Directory Server (ITDS)                  1.5 GB

          Idling guest                                    1.0 GB




      Test   scenarios
        – Leave the guest size fix
        – Decrease the LPAR size in predefined steps to scale the level on memory overcommitment
        – Measure the execution time of a predefined workload (TPM)




10                                                                                                 © 2011 IBM Corporation
Test Results


     Memory – less is better                                    Performance – more is better
                                                                           100
                               BASE settings = 100%
                               • Sum of guest size definition
                               • Base performance
              100%
              100%


               8% saved                                                    106

                               OPTIMAL settings                                       + 6%
                               + Reduce memory by 8%
                               + Improved performance by 6%



                                                                           93
              64% saved        CHEAPEST settings                                      - 7%
                               + Reduce memory by 64%
                               – Decreased performance by 7%


11                                                                               © 2011 IBM Corporation
Results & Recommendation

  Virtual   memory = Physical memory
     → Does not necessarily provide the best performance (at least not for large LPARs, e.g. 28GB)

  Optimal     memory setting: System shows no z/VM paging activity!
     – See PerfKit Panels:
       FCX113 User Paging Activity and Storage Utilization (UPAGE)
       FCX254 Available List Management, by Time (AVAILLOG)

  Estimate    minimum memory requirements:
     – WebSphere Application Server: Sum of all active Java heaps (8GB)

     – DB2:                      Sum of MAX_PARTITION_MEM (2GB), as reported from:
       "SELECT * FROM TABLE (SYSPROC.ADMIN_GET_DBP_MEM_USAGE()) AS T"
       → Value of PEAK_PARTITION_MEM might be used, when highest allocation is captured

     – Linux Page Cache:                Is ignored in our case
       Estimation: Sum of read/write throughput, e.g. 20 MB/sec read and 10 MB/sec write throughput require 30
       MB/sec pages, is ignored in our case in regard to the 10GB contributed from each of WebSphere and DB2

     – Idling guests:                  Can be ignored (no kind of server started!)

     – ITDS:                           Is ignored because of the very small memory footprint

     – Results in 10GB, add 10% Buffer → Minimum System size with good performance was 11GB


12                                                                                                     © 2011 IBM Corporation
Results & Recommendation cont.


  System    runs well in a range from 28 GB to 11 GB
  In  this environment the sum of all active Java heaps (-Xmx) and allocated DB2 memory pools
     define the minimum memory size
     → This is really the lower limit, do not cross!
  Be   aware of the dynamic of a virtualized environment

     New guests are easily created, Java heaps and database pools might be increased without
     notifying the System z administrator

  Monitor   paging activity of your system:
      – FCX113 User Paging Activity and Storage Utilization
      – FCX254 Available List Management → indicates the available memory for new guests
      – FCX135 Average User Wait State Data (→ %PGW)

  Other   workload types/server types might follow similar considerations

  More    details in “Tivoli Provisioning Manager Version 7.1.1.1: Sizing and Capacity Planning”
      – http://public.dhe.ibm.com/software/dw/linux390/perf/ZSW03168USEN.PDF
        Chapter 9. Memory overcommitment




13                                                                                         © 2011 IBM Corporation
Agenda

 Introduction


 z/VM   Resource Manager (SRM)

 Recommendations    for Memory Overcommitment on z/VM

 Large   DCSS

 Summary




14                                                       © 2011 IBM Corporation
DCSS

  Discontiguous   Saved Segments (DCSS) is a z/VM technology used to share memory between a
     number of guests
  The   DCSS itself has been available in z/VM for a while.
     The new feature: The size of a DCSS can now exceed the prior limitation of 2 GB
      – Requires SLES11 SP1 or RHEL 6 Linux distribution

  A DCSS    is memory based,
      – provides very fast access
      – requires real memory

  Allows   sharing of memory pages between guests
      – reduces the memory requirements for the guest, especially when sharing data (read only)
      – The memory is z/VM managed

  Supports     Execute-in-place (XIP), programs code is executed from the DCSS and is not loaded
     into the Linux memory (page cache),
      – Saves Linux memory

  Can   be made persistent, saved to Spool area
      – Requires sufficient Spool space, disk I/O bandwidth of the Spool DASDs can become an important
        performance aspect




15                                                                                                       © 2011 IBM Corporation
DCSS usage


     Where can a DCSS be used:
      As   fast Swap device
      For   sharing read only data
      For   sharing code (e.g. program executables/libraries)

      The
         large DCSS allows the installation of a full middleware stack in the DCSS (WebSphere,
      DB2, etc)
        – The DCSS becomes a consistent unit of one software level, avoids inconsistencies between DCSS and local
          disks

      That's the agenda for the next slides, but before, we start with some basics:
      Types   of DCSS
      How    to create a DCSS
      Access   and load times




16                                                                                                     © 2011 IBM Corporation
Types of DCSS
     The type definition consists of two characters
      The   first character is either E for exclusive access or S for shared access
        – Sharing of storage between virtual machines is based on 1 MB segments, so all 256 pages in any one segment
          must be either exclusive or shared
      Thesecond character of the page descriptor code defines the storage protection
      characteristics of the page. Mostly used in our case
        – N Stands for read/write, no data saved
        – R read-only and saved
        – W read/write access and saved

                                                                       DCSS Type
          Consideration                             SN: Shared R/W           SR: Shared R/O
          File system loaded into DCSS is written
                                                              No                         Yes
          to z/VM spool
          Initial elapsed time to populate the DCSS Faster because no DASD Slower because DASD I/O
          with the files to be shared               I/O is necessary       is required
          Spool processing for DCSS can delay
                                                              No                         Yes
          other spool activity


      Examples:

        – DCSS for swapping:                  SN or EN
        – DCSS for initial population:        SW or EW
        – DCSS for shared code or data:       SR

17                                                                                                      © 2011 IBM Corporation
Defining a DCSS

 From   a CMS user with authentication Class E:
     – Define a DCSS of type SR and size 4 GB in
       2047 MB chunks (units pages)                    4862M
       cp defseg dcss4g1 30000-AFEFF sr                                                    12FDFF
       cp defseg dcss4g2 AFF00-12FDFF sr

     – The CMS guest which wants to save the new                          DCSS
       DCSS needs an appropriate size to include
       the full DCSS size!

     – The spool area must be able to hold three        768M                               30000
       versions of one DCSS (state S, A, P)                             Unused hole
                                                        512M                               20000
 To   edit the DCSS from a linux guest                               Linux guest
     – Add the user directory entry                                   storage size
       NAMESAVE DCSS4G1 DCSS4G2                                  (CP DEF STOR 512M)
                                                                                            0
 To   access the DCSS from a linux guest
                                                                                  Segment address
     – Define the guest with a certain size, e.g. 512 MB
       def stor 512M
     – Ensure that Linux memory and DCSS address range do not overlap



18                                                                                     © 2011 IBM Corporation
Populating a DCSS
     In Linux:
        load the DCSS block device driver:                           modprobe dcssblk
        get the major device number for the dcssblk device:          cat /proc/devices
        create a device node for the DCSS device using the major from above and a free minor
                                          mknod /dev/dcssblk0 b 252 0
        Enable the DCSS via               echo DCSS4G1:DCSS4G2 > /sys/devices/dcssblk/add
         creates a subdirectory with the name of the DCSS within the /sys/devices/dcssblk directory
        Make the DCSS writable by setting the shared value to exclusive use (Type SR is mounted ro per default)
                                         echo 0 > /sys/devices/dcssblk/DCSS4G1/shared
        Make an ext2 file system on the DCSS:                        mke2fs -b 4096 /dev/dcssblk0
        Mount the DCSS:                   mount -t ext2 -o xip /dev/dcssblk0 /mnt
        Copy data to the DCSS
        Unmount the DCSS                 umount /mnt
         no administrative operations possible on a mounted DCSS
        Save the DCSS. The DCSS is so far changed only in memory
         Save to disk (spool):           echo 1 > /sys/devices/dcssblk/DCSS4G1/save
        Remove the DCSS to cause the newly saved DCSS to become active in the spool and purge the old one:
                                        echo "DCSS4G1" > /sys/devices/dcssblk/remove
        Enable the DCSS via               echo DCSS4G1:DCSS4G2 > /sys/devices/dcssblk/add

        Find more information in the Linux device driver book:
         http://www.ibm.com/developerworks/linux/linux390/documentation_dev.html



19                                                                                                     © 2011 IBM Corporation
Loading a DCSS

                                                       Time for initial access of DCSS
                                                                    DCSS type - SR
                                    1000
                                     900
                                     800
                                     700
                                     600
                          Seconds




                                     500
                                     400
                                     300
                                     200
                                     100
                                      0
                                           1   2   3   4   5    6    7   8   9 10 11 12 13 14 15 16

                                                               DCSS size in GB



        echo DCSS4G1:DCSS4G2 > /sys/devices/dcssblk/add
        This may take a while for the first time, be patient!
         Spool file system I/O bandwidth is important here
        Adding the DCSS causes only a minor amount of pages to be allocated (100-200)
        Hits only the first guest, which adds the DCSS. Succeeding guests get it immediately
        Type SN required 5 – 35 seconds

20                                                                                                    © 2011 IBM Corporation
Saving a DCSS

                                          Time to save DCSS after populating
                                                              DCSS type - SR
                                   1800

                                   1600

                                   1400

                                   1200

                                   1000
                         Seconds




                                    800

                                    600

                                    400

                                    200

                                      0
                                          1   2   3   4   5   6   7   8   9 10 11 12 13 14 15 16

                                                          DCSS size in GB




    echo 1 > /sys/devices/dcssblk/DCSS4G1/save                                 This may take a while, be patient!
    Saving a DCSS causes
      – the creation of a new segment (type S, changes to A during save, while the old one changes from A to P)
          • The other guests continue to work with the old version
          • The segment with the type P will be removed once the last guest has released it
      – the allocation of all pages of the DCSS in memory (!)
    Removing the DCSS from the virtual memory of the guest releases the allocated memory
21                                                                                                                   © 2011 IBM Corporation
DCSS as Swap Device - Guest and z/VM was under memory pressure
                                          Throughput by swap device                                                          CPU cost per workload unit by swap device

                           160                                                                                         60%

                           140
                                                                                                                       50%




                                                                                                           CPU cost
Relative throughput [%]




                           120
                                                                                          DISK                         40%
                           100                                                            VDISK
                                                                                          DCSS-EN
                            80                                                            DCSS-SN                      30%
                                                                                          use the memory
                                                                                          for the guest
                            60




                                                                                                           << Better
                                                                                                                       20%
                            40
                                                                                                                       10%
                            20

                             0                                                                                         0%


                         In case a memory based swap device is used, the DCSS provides best throughput and at lowest CPU cost (guest
                          stays in SIE instruction)
                         The best throughput at lowest CPU cost per workload unit was reached with using the storage as guest memory
                           – This setup prevented the need for the guest to swap
                         Recommendation:
                           – When the guest is permanently under memory pressure it is probably better to give it more memory
                           – For short peaks in memory pressure a memory based swap device might be appropriate as fast swapping device
                                 • Should be z/VM based, that the memory pages are allocated only when really in use
                                 • The peak should not appear on all guest at the same time
                                 • Then a DCSS is a good solution: it is fast, with small overhead (smaller than with a disk!)
                   22                                                                                                                                    © 2011 IBM Corporation
Sharing Read Only Data



                           Database location          Normalized throughput

                           Shared minidisk,                    100%
                           - no minidisk cache

                           Shared minidisk                     119%
                           - with minidisk cache

                           DCSS                                991%




      Sharing   a read only database with five guests
      Provides   a throughput improvement of factor 10x compared to a shared disk!
      Recommendation:

        – A DCSS is the first choice for sharing read only data, not updated too often




23                                                                                       © 2011 IBM Corporation
Can we save memory in WebSphere Application Server Environments?

                                                          Identifying memory savings using a DCSS
                                                             Five guests with WebSphere Application Server

                      Throughput - normalized [%]   103

                                                    102
                                                                                                             DCSS – 768 MB
                                                    101
                                                                                                             Disk – 768 MB
                                                    100                                                      Disk – 773 MB
                                                                                                             Disk – 778 MB
                                                    99
                                                                                                             Disk – 783 MB
                                                    98

                                                    97

                                                    96



        Methodology:
          – Installing WebSphere and DB2 in a 6GB DCSS, mounting the DCSS with -o xip
          – Installing WebSphere and DB2 on a shared disk without DCSS (provides in theory 6GB additional memory)
          – z/VM is under memory pressure
     Idea: Scale the guest size without DCSS until throughput from the setup with DCSS case is reached
        Results:
          – Best result with the WebSphere on disk is provided with 5 MB more per guest, but still below the DCSS result
          – Increasing the guests soon causes z/VM paging to DASD and accordingly a performance degradation
            ==> with 50 MB more guest storage the memory pressure is already worse than with the DCSS
          – With only 5 guests we see already a remarkable advantage for the DCSS
              • Saving must be above 25 MB totally for five guest
24                                                                                                                           © 2011 IBM Corporation
Where are the pages from the DCSS?


                                                                DCSS's page location
                  Memory size in MB
                                      450
                                      350
                                      250
                                      150
                                       50
                                      -50
                                       :00


                                             :40


                                                   :20


                                                         :00


                                                               :40


                                                                        :20


                                                                               :00


                                                                                      :40


                                                                                              :20


                                                                                                       :00


                                                                                                             :40


                                                                                                                   :20


                                                                                                                         :00


                                                                                                                               :40
                                             00


                                                   01




                                                                              04


                                                                                     04




                                                                                                                   07


                                                                                                                         08
                                      00




                                                         02


                                                               02


                                                                      03




                                                                                              05


                                                                                                    06


                                                                                                             06




                                                                                                                               08
                                                                           Time (hh:mm)
                                                                    Resident (main storage)    XSTOR




      Looking   at the page location from the pages of the DCSS
        – At no point in time was the whole DCSS in storage, only the used pages
        – 400 MB are allocated in total
        These pages are used once at WebSphere start, but are now migrated because they are not used
        – Central storage contains in average about 10 MB, with a peak allocation of 50 MB
        – DCSS pages are never written to DASD!
      We   are saving at minimum 10 MB per guest
      This
          scales with the amount of guests: for 100 guests 1 GB memory and more are saved when
      using a DCSS


25                                                                                                                                   © 2011 IBM Corporation
Horizontal versus Vertical WAS Guest Stacking – DCSS vs Minidisk
                                            WebSphere JVM stacking                                                                     WebSphere JVM stacking
                                          Throughput with DCSS vs Minidisk                                                           LPAR CPU load with DCSS vs Minidisk
                  4,000
                                                                                                                14
                  3,500
                                                                                                                12
                  3,000
                                                                                                                10
                  2,500
Throughput




                  2,000                                                                                         8




                                                                                                        #IFLs
                  1,500                                                                                         6
                  1,000                                                                                         4
                   500                                                                                          2
                      0                                                                                         0
                          200 190 180 170 160 150 140 130 120 110 100 90 80 70 60 50 40 30 20 10   0                 200 190 180 170 160 150 140 130 120 110 100 90 80 70 60 50 40 30 20 10     0
                                           #JVMs per guest (Total always 200)                                                           #JVMs per guest (Total always 200)
         Guests            1                                  2                 4         10   20 200     Guests 1                                        2                  4          10   20 200

                                               DCSS         shared Minidisk                                                               DCSS         shared Minidisk




                  DCSS or shared minidisks?
                     Impact on throughput is nearly not noticeable
                     Impact on CPU is significant (and as expected)
                           – For small numbers of guests (1 – 4) it is much cheaper to use a minidisk than a DCSS (Savings: 1.4 – 2.2 IFLs)
                           – 10 guests was the break even
                           – With 20 guests and more, the environment with the DCSS needs less CPU (Savings: 1.5 – 2.2 IFLs)




             26                                                                                                                                                              © 2011 IBM Corporation
Agenda

 Introduction


 z/VM   Resource Manager (SRM)

 Recommendations    for Memory Overcommitment on z/VM

 Large   DCSS

 Summary




27                                                       © 2011 IBM Corporation
Memory Overcommitment Summary


  SRM    settings
      – Defaults prevent memory overcommitment for Linux guests (Q3 guests)
      – Adjust LDUBUF and STORBUF settings as required

  Virtual   memory = Physical memory did not provide the best performance
  Optimal    memory settings are as long as there is no z/VM paging activity!
  In  our environment the sum of all active Java heaps and allocated DB2 memory pools defined
     the minimum memory size
     → This is really the lower limit, do not cross!
  Monitor   paging activity of your system:
      – FCX113 User Paging Activity and Storage Utilization
      – FCX254 Available List Management
      – FCX135 Average User Wait State Data (→ %PGW)

  More   details in “Tivoli Provisioning Manager Version 7.1.1.1: Sizing and Capacity Planning”
      – http://public.dhe.ibm.com/software/dw/linux390/perf/ZSW03168USEN.PDF
        Chapter 9. Memory overcommitment




28                                                                                       © 2011 IBM Corporation
DCSS - Summary


  Time   to save/ 1st load (saved DCSS)
     – Do not make the DCSS larger than necessary
     – Consider using a separate guest just for this purpose
     – Moderate update frequency

  Swap   (not saved DCSS)
     – Might be appropriate as swap device for memory peaks

  Sharing   Read Only data
     – A DCSS is the ideal solution for that!

  Sharing   the software stack
     – Installing the whole product in the DCSS provides a consistent source, allows to exchange the DCSS without
       dependency on the Linux guest
     – With 5 guests we already saw an advantage in performance using WebSphere and DB2 in the DCSS
     – With 20 and more guests the CPU cost is lower than a shared minidisk

  More   details at “Large Discontiguous Saved Segments (DCSS) Under Linux”
     – http://public.dhe.ibm.com/software/dw/linux390/perf/ZSW03186USEN.PDF




29                                                                                                     © 2011 IBM Corporation
Backup




30       © 2011 IBM Corporation
z/VM Memory Management


                                                         Active Virtual Memory Pages
     Virtual
     Memory                                               Inactive Virtual Memory Pages


               Gu    Gu    Gu    Gu    Gu
               est   est   est   est   est



                                                         All Active Virtual Memory Pages must be covered with real memory.
                                                         Otherwise the application experiences a page fault when used

                                                         Inactive Virtual Memory Pages might be covered with real memory
     Real      z/    z/
     Memory    V     z/
                     V
               M     V
                     M                           z/      z/VM owned Memory Pages
                     M                           V
                                                 M      Inactive Real Memory




                                             When a guest experiences a page fault for an active page,
                                             → the required page must be brought into real memory from paging space

                                             When all pages are in use and a free page is needed
                                             → a real memory page must be transferred to the paging space




                                  XSTOR                  Active Virtual Memory Pages on Paging Space
     z/VM                                                Inactive Virtual Memory Pages On Paging Space
     Paging
     Devices                                             z/VM owned Memory Pages on Paging Space
                                 Paging
                                 DASD



31                                                                                                                         © 2011 IBM Corporation

Weitere ähnliche Inhalte

Was ist angesagt?

Tivoli Storage Productivity Center... What’s new in v4.2.2?
Tivoli Storage Productivity Center... What’s new in v4.2.2?Tivoli Storage Productivity Center... What’s new in v4.2.2?
Tivoli Storage Productivity Center... What’s new in v4.2.2?IBM India Smarter Computing
 
Supercharge Your Savings and Mainframe Performance
Supercharge Your Savings and Mainframe PerformanceSupercharge Your Savings and Mainframe Performance
Supercharge Your Savings and Mainframe Performancexmeteorite
 
Performing Sandboxed Testing with PlateSpin Forge
Performing Sandboxed Testing with PlateSpin ForgePerforming Sandboxed Testing with PlateSpin Forge
Performing Sandboxed Testing with PlateSpin ForgeNovell
 
Linux on System z Optimizing Resource Utilization for Linux under z/VM – Part II
Linux on System z Optimizing Resource Utilization for Linux under z/VM – Part IILinux on System z Optimizing Resource Utilization for Linux under z/VM – Part II
Linux on System z Optimizing Resource Utilization for Linux under z/VM – Part IIIBM India Smarter Computing
 
Memory Sizing for WebSphere Applications on System z Linux
Memory Sizing for WebSphere Applications on System z LinuxMemory Sizing for WebSphere Applications on System z Linux
Memory Sizing for WebSphere Applications on System z LinuxIBM India Smarter Computing
 
Data center computing trends a survey
Data center computing trends   a surveyData center computing trends   a survey
Data center computing trends a surveyPartha Kundu
 
Windows Server 2012 - Dynamische opslag met Storage Pools
Windows Server 2012 - Dynamische opslag met Storage PoolsWindows Server 2012 - Dynamische opslag met Storage Pools
Windows Server 2012 - Dynamische opslag met Storage PoolsCompuTrain. De IT opleider.
 
Using multi tiered storage systems for storing both structured & unstructured...
Using multi tiered storage systems for storing both structured & unstructured...Using multi tiered storage systems for storing both structured & unstructured...
Using multi tiered storage systems for storing both structured & unstructured...ORACLE USER GROUP ESTONIA
 
Sh pro40 datasheet_se-sbs_2010_05
Sh pro40 datasheet_se-sbs_2010_05Sh pro40 datasheet_se-sbs_2010_05
Sh pro40 datasheet_se-sbs_2010_05hyperlyte3
 
Keeping ChangeMan ZMF environments in perfect shape
Keeping ChangeMan ZMF environments in perfect shapeKeeping ChangeMan ZMF environments in perfect shape
Keeping ChangeMan ZMF environments in perfect shapeAbitMORE SCM
 
Teradata Co-existing Systems Parallel Efficiency -- Calculation & Reconfigura...
Teradata Co-existing Systems Parallel Efficiency -- Calculation & Reconfigura...Teradata Co-existing Systems Parallel Efficiency -- Calculation & Reconfigura...
Teradata Co-existing Systems Parallel Efficiency -- Calculation & Reconfigura...Shaheryar Iqbal
 
Cloud infrastructure licensing_v2
Cloud infrastructure licensing_v2Cloud infrastructure licensing_v2
Cloud infrastructure licensing_v2mikhail.mikheev
 
ABC of Teradata System Performance Analysis
ABC of Teradata System Performance AnalysisABC of Teradata System Performance Analysis
ABC of Teradata System Performance AnalysisShaheryar Iqbal
 
How to achieve better backup with Symantec
How to achieve better backup with SymantecHow to achieve better backup with Symantec
How to achieve better backup with SymantecArrow ECS UK
 

Was ist angesagt? (19)

Tivoli Storage Productivity Center... What’s new in v4.2.2?
Tivoli Storage Productivity Center... What’s new in v4.2.2?Tivoli Storage Productivity Center... What’s new in v4.2.2?
Tivoli Storage Productivity Center... What’s new in v4.2.2?
 
Supercharge Your Savings and Mainframe Performance
Supercharge Your Savings and Mainframe PerformanceSupercharge Your Savings and Mainframe Performance
Supercharge Your Savings and Mainframe Performance
 
ICTM 10
ICTM 10ICTM 10
ICTM 10
 
Performing Sandboxed Testing with PlateSpin Forge
Performing Sandboxed Testing with PlateSpin ForgePerforming Sandboxed Testing with PlateSpin Forge
Performing Sandboxed Testing with PlateSpin Forge
 
Linux on System z Optimizing Resource Utilization for Linux under z/VM – Part II
Linux on System z Optimizing Resource Utilization for Linux under z/VM – Part IILinux on System z Optimizing Resource Utilization for Linux under z/VM – Part II
Linux on System z Optimizing Resource Utilization for Linux under z/VM – Part II
 
Memory Sizing for WebSphere Applications on System z Linux
Memory Sizing for WebSphere Applications on System z LinuxMemory Sizing for WebSphere Applications on System z Linux
Memory Sizing for WebSphere Applications on System z Linux
 
Data center computing trends a survey
Data center computing trends   a surveyData center computing trends   a survey
Data center computing trends a survey
 
XS Boston 2008 ARM
XS Boston 2008 ARMXS Boston 2008 ARM
XS Boston 2008 ARM
 
Windows Server 2012 - Dynamische opslag met Storage Pools
Windows Server 2012 - Dynamische opslag met Storage PoolsWindows Server 2012 - Dynamische opslag met Storage Pools
Windows Server 2012 - Dynamische opslag met Storage Pools
 
Using multi tiered storage systems for storing both structured & unstructured...
Using multi tiered storage systems for storing both structured & unstructured...Using multi tiered storage systems for storing both structured & unstructured...
Using multi tiered storage systems for storing both structured & unstructured...
 
Sh pro40 datasheet_se-sbs_2010_05
Sh pro40 datasheet_se-sbs_2010_05Sh pro40 datasheet_se-sbs_2010_05
Sh pro40 datasheet_se-sbs_2010_05
 
XS Japan 2008 App Data English
XS Japan 2008 App Data EnglishXS Japan 2008 App Data English
XS Japan 2008 App Data English
 
Nakajima numa-final
Nakajima numa-finalNakajima numa-final
Nakajima numa-final
 
Keeping ChangeMan ZMF environments in perfect shape
Keeping ChangeMan ZMF environments in perfect shapeKeeping ChangeMan ZMF environments in perfect shape
Keeping ChangeMan ZMF environments in perfect shape
 
Teradata Co-existing Systems Parallel Efficiency -- Calculation & Reconfigura...
Teradata Co-existing Systems Parallel Efficiency -- Calculation & Reconfigura...Teradata Co-existing Systems Parallel Efficiency -- Calculation & Reconfigura...
Teradata Co-existing Systems Parallel Efficiency -- Calculation & Reconfigura...
 
Cloud infrastructure licensing_v2
Cloud infrastructure licensing_v2Cloud infrastructure licensing_v2
Cloud infrastructure licensing_v2
 
ABC of Teradata System Performance Analysis
ABC of Teradata System Performance AnalysisABC of Teradata System Performance Analysis
ABC of Teradata System Performance Analysis
 
How to achieve better backup with Symantec
How to achieve better backup with SymantecHow to achieve better backup with Symantec
How to achieve better backup with Symantec
 
XS Oracle 2009 Error Detection
XS Oracle 2009 Error DetectionXS Oracle 2009 Error Detection
XS Oracle 2009 Error Detection
 

Andere mochten auch

Linux Memory Basics for SysAdmins - ChinaNetCloud Training
Linux Memory Basics for SysAdmins - ChinaNetCloud TrainingLinux Memory Basics for SysAdmins - ChinaNetCloud Training
Linux Memory Basics for SysAdmins - ChinaNetCloud TrainingChinaNetCloud
 
Linux a free and open source operating system
Linux a free and open source operating systemLinux a free and open source operating system
Linux a free and open source operating systembanwait
 
Linux memorymanagement
Linux memorymanagementLinux memorymanagement
Linux memorymanagementpradeepelinux
 

Andere mochten auch (6)

Linux Memory Basics for SysAdmins - ChinaNetCloud Training
Linux Memory Basics for SysAdmins - ChinaNetCloud TrainingLinux Memory Basics for SysAdmins - ChinaNetCloud Training
Linux Memory Basics for SysAdmins - ChinaNetCloud Training
 
Linux a free and open source operating system
Linux a free and open source operating systemLinux a free and open source operating system
Linux a free and open source operating system
 
OSCh20
OSCh20OSCh20
OSCh20
 
Linux memorymanagement
Linux memorymanagementLinux memorymanagement
Linux memorymanagement
 
Linux memory
Linux memoryLinux memory
Linux memory
 
Linux Memory
Linux MemoryLinux Memory
Linux Memory
 

Ähnlich wie Linux on System z Optimizing Resource Utilization for Linux under z/VM - Part1

Session 7362 Handout 427 0
Session 7362 Handout 427 0Session 7362 Handout 427 0
Session 7362 Handout 427 0jln1028
 
VMworld 2013: Performance and Capacity Management of DRS Clusters
VMworld 2013: Performance and Capacity Management of DRS Clusters VMworld 2013: Performance and Capacity Management of DRS Clusters
VMworld 2013: Performance and Capacity Management of DRS Clusters VMworld
 
Connect2013 id506 hadr ideas for social business
Connect2013 id506 hadr ideas for social businessConnect2013 id506 hadr ideas for social business
Connect2013 id506 hadr ideas for social businessLuis Guirigay
 
DB2 Design for High Availability and Scalability
DB2 Design for High Availability and ScalabilityDB2 Design for High Availability and Scalability
DB2 Design for High Availability and ScalabilitySurekha Parekh
 
The benefits of IBM FlashSystems
The benefits of IBM FlashSystemsThe benefits of IBM FlashSystems
The benefits of IBM FlashSystemsLuca Comparini
 
Id105 fortify your ibm lotus notes and ibm lotus domino infrastructure agai...
Id105   fortify your ibm lotus notes and ibm lotus domino infrastructure agai...Id105   fortify your ibm lotus notes and ibm lotus domino infrastructure agai...
Id105 fortify your ibm lotus notes and ibm lotus domino infrastructure agai...waukema
 
A presentaion on Panasas HPC NAS
A presentaion on Panasas HPC NASA presentaion on Panasas HPC NAS
A presentaion on Panasas HPC NASRahul Janghel
 
We4IT lcty 2013 - infra-man - domino run faster
We4IT lcty 2013 - infra-man - domino run faster We4IT lcty 2013 - infra-man - domino run faster
We4IT lcty 2013 - infra-man - domino run faster We4IT Group
 
Controlling Memory Footprint at All Layers: Linux Kernel, Applications, Libra...
Controlling Memory Footprint at All Layers: Linux Kernel, Applications, Libra...Controlling Memory Footprint at All Layers: Linux Kernel, Applications, Libra...
Controlling Memory Footprint at All Layers: Linux Kernel, Applications, Libra...peknap
 
Stackops - Openstack Nova sizing & service definition
Stackops - Openstack Nova sizing & service definitionStackops - Openstack Nova sizing & service definition
Stackops - Openstack Nova sizing & service definitionStackops
 
SanDisk: Persistent Memory and Cassandra
SanDisk: Persistent Memory and CassandraSanDisk: Persistent Memory and Cassandra
SanDisk: Persistent Memory and CassandraDataStax Academy
 
IBM Upgrades SVC with Solid State Drives — Achieves Better Storage Utilization
IBM Upgrades SVC with Solid State Drives — Achieves Better Storage UtilizationIBM Upgrades SVC with Solid State Drives — Achieves Better Storage Utilization
IBM Upgrades SVC with Solid State Drives — Achieves Better Storage UtilizationIBM India Smarter Computing
 
Tips and Tricks for SAP Sybase IQ
Tips and Tricks for SAP  Sybase IQTips and Tricks for SAP  Sybase IQ
Tips and Tricks for SAP Sybase IQDon Brizendine
 
Webcenter application performance tuning guide
Webcenter application performance tuning guideWebcenter application performance tuning guide
Webcenter application performance tuning guideVinay Kumar
 
DB2 pureScale Overview Sept 2010
DB2 pureScale Overview Sept 2010DB2 pureScale Overview Sept 2010
DB2 pureScale Overview Sept 2010Laura Hood
 
Transforming your Business with Scale-Out Flash: How MongoDB & Flash Accelera...
Transforming your Business with Scale-Out Flash: How MongoDB & Flash Accelera...Transforming your Business with Scale-Out Flash: How MongoDB & Flash Accelera...
Transforming your Business with Scale-Out Flash: How MongoDB & Flash Accelera...MongoDB
 
The Best Storage For V Mware Environments Customer Presentation Jul201
The Best Storage For V Mware Environments Customer Presentation Jul201The Best Storage For V Mware Environments Customer Presentation Jul201
The Best Storage For V Mware Environments Customer Presentation Jul201Michael Hudak
 
z/VM Performance Analysis
z/VM Performance Analysisz/VM Performance Analysis
z/VM Performance AnalysisRodrigo Campos
 
IBM flash systems
IBM flash systems IBM flash systems
IBM flash systems Solv AS
 
Hyper-V Best Practices & Tips and Tricks
Hyper-V Best Practices & Tips and TricksHyper-V Best Practices & Tips and Tricks
Hyper-V Best Practices & Tips and TricksAmit Gatenyo
 

Ähnlich wie Linux on System z Optimizing Resource Utilization for Linux under z/VM - Part1 (20)

Session 7362 Handout 427 0
Session 7362 Handout 427 0Session 7362 Handout 427 0
Session 7362 Handout 427 0
 
VMworld 2013: Performance and Capacity Management of DRS Clusters
VMworld 2013: Performance and Capacity Management of DRS Clusters VMworld 2013: Performance and Capacity Management of DRS Clusters
VMworld 2013: Performance and Capacity Management of DRS Clusters
 
Connect2013 id506 hadr ideas for social business
Connect2013 id506 hadr ideas for social businessConnect2013 id506 hadr ideas for social business
Connect2013 id506 hadr ideas for social business
 
DB2 Design for High Availability and Scalability
DB2 Design for High Availability and ScalabilityDB2 Design for High Availability and Scalability
DB2 Design for High Availability and Scalability
 
The benefits of IBM FlashSystems
The benefits of IBM FlashSystemsThe benefits of IBM FlashSystems
The benefits of IBM FlashSystems
 
Id105 fortify your ibm lotus notes and ibm lotus domino infrastructure agai...
Id105   fortify your ibm lotus notes and ibm lotus domino infrastructure agai...Id105   fortify your ibm lotus notes and ibm lotus domino infrastructure agai...
Id105 fortify your ibm lotus notes and ibm lotus domino infrastructure agai...
 
A presentaion on Panasas HPC NAS
A presentaion on Panasas HPC NASA presentaion on Panasas HPC NAS
A presentaion on Panasas HPC NAS
 
We4IT lcty 2013 - infra-man - domino run faster
We4IT lcty 2013 - infra-man - domino run faster We4IT lcty 2013 - infra-man - domino run faster
We4IT lcty 2013 - infra-man - domino run faster
 
Controlling Memory Footprint at All Layers: Linux Kernel, Applications, Libra...
Controlling Memory Footprint at All Layers: Linux Kernel, Applications, Libra...Controlling Memory Footprint at All Layers: Linux Kernel, Applications, Libra...
Controlling Memory Footprint at All Layers: Linux Kernel, Applications, Libra...
 
Stackops - Openstack Nova sizing & service definition
Stackops - Openstack Nova sizing & service definitionStackops - Openstack Nova sizing & service definition
Stackops - Openstack Nova sizing & service definition
 
SanDisk: Persistent Memory and Cassandra
SanDisk: Persistent Memory and CassandraSanDisk: Persistent Memory and Cassandra
SanDisk: Persistent Memory and Cassandra
 
IBM Upgrades SVC with Solid State Drives — Achieves Better Storage Utilization
IBM Upgrades SVC with Solid State Drives — Achieves Better Storage UtilizationIBM Upgrades SVC with Solid State Drives — Achieves Better Storage Utilization
IBM Upgrades SVC with Solid State Drives — Achieves Better Storage Utilization
 
Tips and Tricks for SAP Sybase IQ
Tips and Tricks for SAP  Sybase IQTips and Tricks for SAP  Sybase IQ
Tips and Tricks for SAP Sybase IQ
 
Webcenter application performance tuning guide
Webcenter application performance tuning guideWebcenter application performance tuning guide
Webcenter application performance tuning guide
 
DB2 pureScale Overview Sept 2010
DB2 pureScale Overview Sept 2010DB2 pureScale Overview Sept 2010
DB2 pureScale Overview Sept 2010
 
Transforming your Business with Scale-Out Flash: How MongoDB & Flash Accelera...
Transforming your Business with Scale-Out Flash: How MongoDB & Flash Accelera...Transforming your Business with Scale-Out Flash: How MongoDB & Flash Accelera...
Transforming your Business with Scale-Out Flash: How MongoDB & Flash Accelera...
 
The Best Storage For V Mware Environments Customer Presentation Jul201
The Best Storage For V Mware Environments Customer Presentation Jul201The Best Storage For V Mware Environments Customer Presentation Jul201
The Best Storage For V Mware Environments Customer Presentation Jul201
 
z/VM Performance Analysis
z/VM Performance Analysisz/VM Performance Analysis
z/VM Performance Analysis
 
IBM flash systems
IBM flash systems IBM flash systems
IBM flash systems
 
Hyper-V Best Practices & Tips and Tricks
Hyper-V Best Practices & Tips and TricksHyper-V Best Practices & Tips and Tricks
Hyper-V Best Practices & Tips and Tricks
 

Mehr von IBM India Smarter Computing

Using the IBM XIV Storage System in OpenStack Cloud Environments
Using the IBM XIV Storage System in OpenStack Cloud Environments Using the IBM XIV Storage System in OpenStack Cloud Environments
Using the IBM XIV Storage System in OpenStack Cloud Environments IBM India Smarter Computing
 
TSL03104USEN Exploring VMware vSphere Storage API for Array Integration on th...
TSL03104USEN Exploring VMware vSphere Storage API for Array Integration on th...TSL03104USEN Exploring VMware vSphere Storage API for Array Integration on th...
TSL03104USEN Exploring VMware vSphere Storage API for Array Integration on th...IBM India Smarter Computing
 
A Comparison of PowerVM and Vmware Virtualization Performance
A Comparison of PowerVM and Vmware Virtualization PerformanceA Comparison of PowerVM and Vmware Virtualization Performance
A Comparison of PowerVM and Vmware Virtualization PerformanceIBM India Smarter Computing
 
IBM pureflex system and vmware vcloud enterprise suite reference architecture
IBM pureflex system and vmware vcloud enterprise suite reference architectureIBM pureflex system and vmware vcloud enterprise suite reference architecture
IBM pureflex system and vmware vcloud enterprise suite reference architectureIBM India Smarter Computing
 

Mehr von IBM India Smarter Computing (20)

Using the IBM XIV Storage System in OpenStack Cloud Environments
Using the IBM XIV Storage System in OpenStack Cloud Environments Using the IBM XIV Storage System in OpenStack Cloud Environments
Using the IBM XIV Storage System in OpenStack Cloud Environments
 
All-flash Needs End to End Storage Efficiency
All-flash Needs End to End Storage EfficiencyAll-flash Needs End to End Storage Efficiency
All-flash Needs End to End Storage Efficiency
 
TSL03104USEN Exploring VMware vSphere Storage API for Array Integration on th...
TSL03104USEN Exploring VMware vSphere Storage API for Array Integration on th...TSL03104USEN Exploring VMware vSphere Storage API for Array Integration on th...
TSL03104USEN Exploring VMware vSphere Storage API for Array Integration on th...
 
IBM FlashSystem 840 Product Guide
IBM FlashSystem 840 Product GuideIBM FlashSystem 840 Product Guide
IBM FlashSystem 840 Product Guide
 
IBM System x3250 M5
IBM System x3250 M5IBM System x3250 M5
IBM System x3250 M5
 
IBM NeXtScale nx360 M4
IBM NeXtScale nx360 M4IBM NeXtScale nx360 M4
IBM NeXtScale nx360 M4
 
IBM System x3650 M4 HD
IBM System x3650 M4 HDIBM System x3650 M4 HD
IBM System x3650 M4 HD
 
IBM System x3300 M4
IBM System x3300 M4IBM System x3300 M4
IBM System x3300 M4
 
IBM System x iDataPlex dx360 M4
IBM System x iDataPlex dx360 M4IBM System x iDataPlex dx360 M4
IBM System x iDataPlex dx360 M4
 
IBM System x3500 M4
IBM System x3500 M4IBM System x3500 M4
IBM System x3500 M4
 
IBM System x3550 M4
IBM System x3550 M4IBM System x3550 M4
IBM System x3550 M4
 
IBM System x3650 M4
IBM System x3650 M4IBM System x3650 M4
IBM System x3650 M4
 
IBM System x3500 M3
IBM System x3500 M3IBM System x3500 M3
IBM System x3500 M3
 
IBM System x3400 M3
IBM System x3400 M3IBM System x3400 M3
IBM System x3400 M3
 
IBM System x3250 M3
IBM System x3250 M3IBM System x3250 M3
IBM System x3250 M3
 
IBM System x3200 M3
IBM System x3200 M3IBM System x3200 M3
IBM System x3200 M3
 
IBM PowerVC Introduction and Configuration
IBM PowerVC Introduction and ConfigurationIBM PowerVC Introduction and Configuration
IBM PowerVC Introduction and Configuration
 
A Comparison of PowerVM and Vmware Virtualization Performance
A Comparison of PowerVM and Vmware Virtualization PerformanceA Comparison of PowerVM and Vmware Virtualization Performance
A Comparison of PowerVM and Vmware Virtualization Performance
 
IBM pureflex system and vmware vcloud enterprise suite reference architecture
IBM pureflex system and vmware vcloud enterprise suite reference architectureIBM pureflex system and vmware vcloud enterprise suite reference architecture
IBM pureflex system and vmware vcloud enterprise suite reference architecture
 
X6: The sixth generation of EXA Technology
X6: The sixth generation of EXA TechnologyX6: The sixth generation of EXA Technology
X6: The sixth generation of EXA Technology
 

Linux on System z Optimizing Resource Utilization for Linux under z/VM - Part1

  • 1. 2011-10-25 Linux on System z Optimizing Resource Utilization for Linux under z/VM - Part1 Dr. Juergen Doelle IBM Germany Research & Development visit us at http://www.ibm.com/developerworks/linux/linux390/perf/index.html © 2011 IBM Corporation
  • 2. Trademarks IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International Business Machines Corp., registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at “Copyright and trademark information” at www.ibm.com/legal/copytrade.shtml. Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both. Java and all Java-based trademarks and logos are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both. Other product and service names might be trademarks of IBM or other companies. 2 © 2011 IBM Corporation
  • 3. Agenda Introduction z/VM Resource Manager (SRM) Recommendations for Memory Overcommitment on z/VM Large DCSS Summary 3 © 2011 IBM Corporation
  • 4. Introduction  Running virtualized guests provides significant advantages in regard of – Systems management – Resource utilization  Running many systems with shared resources introduces a new dynamic into system compared to one system on one box  Important questions – Has the right system the right resources at the right moment? – Can I bring more guests online or increase guest sizes? – When does overcommitment become counterproductive?  When are the service level goals not longer met? – Which load situation is acceptable?  Objectives of that presentation – What can be done to optimize resource consumption, what are the options – Impact on WebSphere Application Sever environments and databases – Focus is on memory utilization 4 © 2011 IBM Corporation
  • 5. Agenda Introduction z/VM Resource Manager (SRM) Recommendations for Memory Overcommitment on z/VM Large DCSS Summary 5 © 2011 IBM Corporation
  • 6. SRM Considerations for Linux guests  The responsibility of the SRM is: – to ensure that guests have the required resources available when scheduled – Guests are divided into queues Q1 – Q3 according to their interactivity • CMS guests are typically Q1 guests • Linux guests are typically Q3 guests – When a guest should be scheduled and SRM comes to the decision that the intended amount of resources can not be granted: ==> the guest is placed on the eligible list and waits  SRM settings might be overruled by enabling QUICKDISPATCH for the guests – But this should be used carefully, e.g. for business critical guests – Do not use QUICKDISPATCH for the majority of the guests, this limits z/VM's possibilities to dynamically manage the resource usage  Consider that Linux guests are running mostly in Q3! 6 © 2011 IBM Corporation
  • 7. SRM Considerations for Linux guests  Default SRM settings (q srm) – LDUBUF : Q1=100% Q2=75% Q3=60% – STORBUF: Q1=125% Q2=105% Q3=95%  LDUBUF – to partition the system's paging resources – 60% for Q3 means: All Q3 guests together must use at maximum 60% of the paging resources if already used → eligible list – Recommendation: SET SRM LDUBUF 100 100 100 to allow all Q3 guests to allocate the whole paging space  STORBUF – to partition host storage (central storage) – 95% for Q3 means: All Q3 guests together must use only 95% of the system storage → This prevents memory overcommitment when running Linux guests – Recommendation: SET SRM storbuf 300 250 200 to allow all Q3 guests to allocate twice the amount of real storage – Depending on the level of overcommitment and amount of active/inactive guests, it might be necessary to go even higher, e.g. SET SRM storbuf 300 300 300 – Ensure to have sufficient paging space! 7 © 2011 IBM Corporation
  • 8. Agenda Introduction z/VM Resource Manager (SRM) Recommendations for Memory Overcommitment on z/VM Large DCSS Summary 8 © 2011 IBM Corporation
  • 9. Memory overcommitment  Memory overcommitment is often mentioned as a major benefit of virtualized environments Memory overcommitment is the ability to use more virtual memory as physically available memory. It is based on the assumption that not all guests use their memory at the same time and/or some guests are over-sized in their memory setup.  Recommendations / limits are missing Very different definitions exist for the level of memory overcommitment – they are independent of the used middle ware – “Common” System z ratio is 3:1 virtual (guest memory) to physical memory E.g. run guests defined with a total of 3GB on 1 GB real memory – Performance can be heavily degraded when the memory overcommitment level is to high  Goal: Proved memory – overcommitment recommendations  Identify “rules” – Determine the minimum amount of physical memory required to run with an acceptable performance – Identify a dependency on the used middle ware / applications 9 © 2011 IBM Corporation
  • 10. Tested Scenario  Test environment – Running a mix of server types as Linux guests on z/VM: LPAR with 28GB central storage + 2 GB expanded storage Guest workload Guest Memory WebSphere Application Server 13.5 GB (Java heaps 8GB) Database DB2 12.0 GB (memory pools about 2 GB) Tivoli Directory Server (ITDS) 1.5 GB Idling guest 1.0 GB  Test scenarios – Leave the guest size fix – Decrease the LPAR size in predefined steps to scale the level on memory overcommitment – Measure the execution time of a predefined workload (TPM) 10 © 2011 IBM Corporation
  • 11. Test Results Memory – less is better Performance – more is better 100 BASE settings = 100% • Sum of guest size definition • Base performance 100% 100% 8% saved 106 OPTIMAL settings + 6% + Reduce memory by 8% + Improved performance by 6% 93 64% saved CHEAPEST settings - 7% + Reduce memory by 64% – Decreased performance by 7% 11 © 2011 IBM Corporation
  • 12. Results & Recommendation  Virtual memory = Physical memory → Does not necessarily provide the best performance (at least not for large LPARs, e.g. 28GB)  Optimal memory setting: System shows no z/VM paging activity! – See PerfKit Panels: FCX113 User Paging Activity and Storage Utilization (UPAGE) FCX254 Available List Management, by Time (AVAILLOG)  Estimate minimum memory requirements: – WebSphere Application Server: Sum of all active Java heaps (8GB) – DB2: Sum of MAX_PARTITION_MEM (2GB), as reported from: "SELECT * FROM TABLE (SYSPROC.ADMIN_GET_DBP_MEM_USAGE()) AS T" → Value of PEAK_PARTITION_MEM might be used, when highest allocation is captured – Linux Page Cache: Is ignored in our case Estimation: Sum of read/write throughput, e.g. 20 MB/sec read and 10 MB/sec write throughput require 30 MB/sec pages, is ignored in our case in regard to the 10GB contributed from each of WebSphere and DB2 – Idling guests: Can be ignored (no kind of server started!) – ITDS: Is ignored because of the very small memory footprint – Results in 10GB, add 10% Buffer → Minimum System size with good performance was 11GB 12 © 2011 IBM Corporation
  • 13. Results & Recommendation cont.  System runs well in a range from 28 GB to 11 GB  In this environment the sum of all active Java heaps (-Xmx) and allocated DB2 memory pools define the minimum memory size → This is really the lower limit, do not cross!  Be aware of the dynamic of a virtualized environment New guests are easily created, Java heaps and database pools might be increased without notifying the System z administrator  Monitor paging activity of your system: – FCX113 User Paging Activity and Storage Utilization – FCX254 Available List Management → indicates the available memory for new guests – FCX135 Average User Wait State Data (→ %PGW)  Other workload types/server types might follow similar considerations  More details in “Tivoli Provisioning Manager Version 7.1.1.1: Sizing and Capacity Planning” – http://public.dhe.ibm.com/software/dw/linux390/perf/ZSW03168USEN.PDF Chapter 9. Memory overcommitment 13 © 2011 IBM Corporation
  • 14. Agenda Introduction z/VM Resource Manager (SRM) Recommendations for Memory Overcommitment on z/VM Large DCSS Summary 14 © 2011 IBM Corporation
  • 15. DCSS  Discontiguous Saved Segments (DCSS) is a z/VM technology used to share memory between a number of guests  The DCSS itself has been available in z/VM for a while. The new feature: The size of a DCSS can now exceed the prior limitation of 2 GB – Requires SLES11 SP1 or RHEL 6 Linux distribution  A DCSS is memory based, – provides very fast access – requires real memory  Allows sharing of memory pages between guests – reduces the memory requirements for the guest, especially when sharing data (read only) – The memory is z/VM managed  Supports Execute-in-place (XIP), programs code is executed from the DCSS and is not loaded into the Linux memory (page cache), – Saves Linux memory  Can be made persistent, saved to Spool area – Requires sufficient Spool space, disk I/O bandwidth of the Spool DASDs can become an important performance aspect 15 © 2011 IBM Corporation
  • 16. DCSS usage Where can a DCSS be used:  As fast Swap device  For sharing read only data  For sharing code (e.g. program executables/libraries)  The large DCSS allows the installation of a full middleware stack in the DCSS (WebSphere, DB2, etc) – The DCSS becomes a consistent unit of one software level, avoids inconsistencies between DCSS and local disks That's the agenda for the next slides, but before, we start with some basics:  Types of DCSS  How to create a DCSS  Access and load times 16 © 2011 IBM Corporation
  • 17. Types of DCSS The type definition consists of two characters  The first character is either E for exclusive access or S for shared access – Sharing of storage between virtual machines is based on 1 MB segments, so all 256 pages in any one segment must be either exclusive or shared  Thesecond character of the page descriptor code defines the storage protection characteristics of the page. Mostly used in our case – N Stands for read/write, no data saved – R read-only and saved – W read/write access and saved DCSS Type Consideration SN: Shared R/W SR: Shared R/O File system loaded into DCSS is written No Yes to z/VM spool Initial elapsed time to populate the DCSS Faster because no DASD Slower because DASD I/O with the files to be shared I/O is necessary is required Spool processing for DCSS can delay No Yes other spool activity  Examples: – DCSS for swapping: SN or EN – DCSS for initial population: SW or EW – DCSS for shared code or data: SR 17 © 2011 IBM Corporation
  • 18. Defining a DCSS  From a CMS user with authentication Class E: – Define a DCSS of type SR and size 4 GB in 2047 MB chunks (units pages) 4862M cp defseg dcss4g1 30000-AFEFF sr 12FDFF cp defseg dcss4g2 AFF00-12FDFF sr – The CMS guest which wants to save the new DCSS DCSS needs an appropriate size to include the full DCSS size! – The spool area must be able to hold three 768M 30000 versions of one DCSS (state S, A, P) Unused hole 512M 20000  To edit the DCSS from a linux guest Linux guest – Add the user directory entry storage size NAMESAVE DCSS4G1 DCSS4G2 (CP DEF STOR 512M) 0  To access the DCSS from a linux guest Segment address – Define the guest with a certain size, e.g. 512 MB def stor 512M – Ensure that Linux memory and DCSS address range do not overlap 18 © 2011 IBM Corporation
  • 19. Populating a DCSS In Linux:  load the DCSS block device driver: modprobe dcssblk  get the major device number for the dcssblk device: cat /proc/devices  create a device node for the DCSS device using the major from above and a free minor mknod /dev/dcssblk0 b 252 0  Enable the DCSS via echo DCSS4G1:DCSS4G2 > /sys/devices/dcssblk/add creates a subdirectory with the name of the DCSS within the /sys/devices/dcssblk directory  Make the DCSS writable by setting the shared value to exclusive use (Type SR is mounted ro per default) echo 0 > /sys/devices/dcssblk/DCSS4G1/shared  Make an ext2 file system on the DCSS: mke2fs -b 4096 /dev/dcssblk0  Mount the DCSS: mount -t ext2 -o xip /dev/dcssblk0 /mnt  Copy data to the DCSS  Unmount the DCSS umount /mnt no administrative operations possible on a mounted DCSS  Save the DCSS. The DCSS is so far changed only in memory Save to disk (spool): echo 1 > /sys/devices/dcssblk/DCSS4G1/save  Remove the DCSS to cause the newly saved DCSS to become active in the spool and purge the old one: echo "DCSS4G1" > /sys/devices/dcssblk/remove  Enable the DCSS via echo DCSS4G1:DCSS4G2 > /sys/devices/dcssblk/add  Find more information in the Linux device driver book: http://www.ibm.com/developerworks/linux/linux390/documentation_dev.html 19 © 2011 IBM Corporation
  • 20. Loading a DCSS Time for initial access of DCSS DCSS type - SR 1000 900 800 700 600 Seconds 500 400 300 200 100 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 DCSS size in GB  echo DCSS4G1:DCSS4G2 > /sys/devices/dcssblk/add  This may take a while for the first time, be patient! Spool file system I/O bandwidth is important here  Adding the DCSS causes only a minor amount of pages to be allocated (100-200)  Hits only the first guest, which adds the DCSS. Succeeding guests get it immediately  Type SN required 5 – 35 seconds 20 © 2011 IBM Corporation
  • 21. Saving a DCSS Time to save DCSS after populating DCSS type - SR 1800 1600 1400 1200 1000 Seconds 800 600 400 200 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 DCSS size in GB  echo 1 > /sys/devices/dcssblk/DCSS4G1/save This may take a while, be patient!  Saving a DCSS causes – the creation of a new segment (type S, changes to A during save, while the old one changes from A to P) • The other guests continue to work with the old version • The segment with the type P will be removed once the last guest has released it – the allocation of all pages of the DCSS in memory (!)  Removing the DCSS from the virtual memory of the guest releases the allocated memory 21 © 2011 IBM Corporation
  • 22. DCSS as Swap Device - Guest and z/VM was under memory pressure Throughput by swap device CPU cost per workload unit by swap device 160 60% 140 50% CPU cost Relative throughput [%] 120 DISK 40% 100 VDISK DCSS-EN 80 DCSS-SN 30% use the memory for the guest 60 << Better 20% 40 10% 20 0 0%  In case a memory based swap device is used, the DCSS provides best throughput and at lowest CPU cost (guest stays in SIE instruction)  The best throughput at lowest CPU cost per workload unit was reached with using the storage as guest memory – This setup prevented the need for the guest to swap  Recommendation: – When the guest is permanently under memory pressure it is probably better to give it more memory – For short peaks in memory pressure a memory based swap device might be appropriate as fast swapping device • Should be z/VM based, that the memory pages are allocated only when really in use • The peak should not appear on all guest at the same time • Then a DCSS is a good solution: it is fast, with small overhead (smaller than with a disk!) 22 © 2011 IBM Corporation
  • 23. Sharing Read Only Data Database location Normalized throughput Shared minidisk, 100% - no minidisk cache Shared minidisk 119% - with minidisk cache DCSS 991%  Sharing a read only database with five guests  Provides a throughput improvement of factor 10x compared to a shared disk!  Recommendation: – A DCSS is the first choice for sharing read only data, not updated too often 23 © 2011 IBM Corporation
  • 24. Can we save memory in WebSphere Application Server Environments? Identifying memory savings using a DCSS Five guests with WebSphere Application Server Throughput - normalized [%] 103 102 DCSS – 768 MB 101 Disk – 768 MB 100 Disk – 773 MB Disk – 778 MB 99 Disk – 783 MB 98 97 96  Methodology: – Installing WebSphere and DB2 in a 6GB DCSS, mounting the DCSS with -o xip – Installing WebSphere and DB2 on a shared disk without DCSS (provides in theory 6GB additional memory) – z/VM is under memory pressure Idea: Scale the guest size without DCSS until throughput from the setup with DCSS case is reached  Results: – Best result with the WebSphere on disk is provided with 5 MB more per guest, but still below the DCSS result – Increasing the guests soon causes z/VM paging to DASD and accordingly a performance degradation ==> with 50 MB more guest storage the memory pressure is already worse than with the DCSS – With only 5 guests we see already a remarkable advantage for the DCSS • Saving must be above 25 MB totally for five guest 24 © 2011 IBM Corporation
  • 25. Where are the pages from the DCSS? DCSS's page location Memory size in MB 450 350 250 150 50 -50 :00 :40 :20 :00 :40 :20 :00 :40 :20 :00 :40 :20 :00 :40 00 01 04 04 07 08 00 02 02 03 05 06 06 08 Time (hh:mm) Resident (main storage) XSTOR  Looking at the page location from the pages of the DCSS – At no point in time was the whole DCSS in storage, only the used pages – 400 MB are allocated in total These pages are used once at WebSphere start, but are now migrated because they are not used – Central storage contains in average about 10 MB, with a peak allocation of 50 MB – DCSS pages are never written to DASD!  We are saving at minimum 10 MB per guest  This scales with the amount of guests: for 100 guests 1 GB memory and more are saved when using a DCSS 25 © 2011 IBM Corporation
  • 26. Horizontal versus Vertical WAS Guest Stacking – DCSS vs Minidisk WebSphere JVM stacking WebSphere JVM stacking Throughput with DCSS vs Minidisk LPAR CPU load with DCSS vs Minidisk 4,000 14 3,500 12 3,000 10 2,500 Throughput 2,000 8 #IFLs 1,500 6 1,000 4 500 2 0 0 200 190 180 170 160 150 140 130 120 110 100 90 80 70 60 50 40 30 20 10 0 200 190 180 170 160 150 140 130 120 110 100 90 80 70 60 50 40 30 20 10 0 #JVMs per guest (Total always 200) #JVMs per guest (Total always 200) Guests 1 2 4 10 20 200 Guests 1 2 4 10 20 200 DCSS shared Minidisk DCSS shared Minidisk DCSS or shared minidisks?  Impact on throughput is nearly not noticeable  Impact on CPU is significant (and as expected) – For small numbers of guests (1 – 4) it is much cheaper to use a minidisk than a DCSS (Savings: 1.4 – 2.2 IFLs) – 10 guests was the break even – With 20 guests and more, the environment with the DCSS needs less CPU (Savings: 1.5 – 2.2 IFLs) 26 © 2011 IBM Corporation
  • 27. Agenda Introduction z/VM Resource Manager (SRM) Recommendations for Memory Overcommitment on z/VM Large DCSS Summary 27 © 2011 IBM Corporation
  • 28. Memory Overcommitment Summary  SRM settings – Defaults prevent memory overcommitment for Linux guests (Q3 guests) – Adjust LDUBUF and STORBUF settings as required  Virtual memory = Physical memory did not provide the best performance  Optimal memory settings are as long as there is no z/VM paging activity!  In our environment the sum of all active Java heaps and allocated DB2 memory pools defined the minimum memory size → This is really the lower limit, do not cross!  Monitor paging activity of your system: – FCX113 User Paging Activity and Storage Utilization – FCX254 Available List Management – FCX135 Average User Wait State Data (→ %PGW)  More details in “Tivoli Provisioning Manager Version 7.1.1.1: Sizing and Capacity Planning” – http://public.dhe.ibm.com/software/dw/linux390/perf/ZSW03168USEN.PDF Chapter 9. Memory overcommitment 28 © 2011 IBM Corporation
  • 29. DCSS - Summary  Time to save/ 1st load (saved DCSS) – Do not make the DCSS larger than necessary – Consider using a separate guest just for this purpose – Moderate update frequency  Swap (not saved DCSS) – Might be appropriate as swap device for memory peaks  Sharing Read Only data – A DCSS is the ideal solution for that!  Sharing the software stack – Installing the whole product in the DCSS provides a consistent source, allows to exchange the DCSS without dependency on the Linux guest – With 5 guests we already saw an advantage in performance using WebSphere and DB2 in the DCSS – With 20 and more guests the CPU cost is lower than a shared minidisk  More details at “Large Discontiguous Saved Segments (DCSS) Under Linux” – http://public.dhe.ibm.com/software/dw/linux390/perf/ZSW03186USEN.PDF 29 © 2011 IBM Corporation
  • 30. Backup 30 © 2011 IBM Corporation
  • 31. z/VM Memory Management Active Virtual Memory Pages Virtual Memory Inactive Virtual Memory Pages Gu Gu Gu Gu Gu est est est est est All Active Virtual Memory Pages must be covered with real memory. Otherwise the application experiences a page fault when used Inactive Virtual Memory Pages might be covered with real memory Real z/ z/ Memory V z/ V M V M z/ z/VM owned Memory Pages M V M Inactive Real Memory When a guest experiences a page fault for an active page, → the required page must be brought into real memory from paging space When all pages are in use and a free page is needed → a real memory page must be transferred to the paging space XSTOR Active Virtual Memory Pages on Paging Space z/VM Inactive Virtual Memory Pages On Paging Space Paging Devices z/VM owned Memory Pages on Paging Space Paging DASD 31 © 2011 IBM Corporation