abount paging and swapping

(or Pagefiles vs. Swap / Swapfiles)

Swapfiles

Source: lkml.iu.edu/hypermail/linux/kernel/9907.2/0462.html

Linux will swap existing pages to disk on a per process basis. In principle, a process might allocate 1 Gb of buffer, but always be using the same one physical page.

Source: stackoverflow.com

The advantage of this is that it is relatively simple to grasp and memory for a program is always allocated contiguously, the downside is that performance on a machine can become absolutely abysmal when the system ends up in a state where things are constantly swapping. The algorithm also involves the repeated swapping in and out of data that will not be used in the foreseeable future.

(…)

With swapping, parts of memory which are not in use are written to disk; this enables one to run several programs whose total memory consumption is greater than the amount of physical memory. When a program makes a request for a part of memory that was written to the disk, that part has to be loaded into memory. To make room for it, another part has to be written to the disk (effectively the two parts swap places – hence the name). This “extension” of physical memory is generally known as “virtual memory”.

(…)

Modern systems use both paging and swapping, and pages are what is being swapped in and out of memory.

Pagefiles

Source: techarp.com

In contrast, paging files function by moving “pages” of a program from system memory into the paging file. These pages are 4KB in size. The entire program does not get swapped wholesale into the paging file.

While swapping occurs when there is heavy demand on the system memory, paging can occur preemptively. This means that the operating system can page out parts of a program when it is minimized or left idle for some time.

Swapfiles were used in old iterations of Microsoft Windows, prior to Windows 95. From Windows 95 onwards, all Windows versions use only paging files.

Source: helpdeskgeek.com

Windows 8 supports both paging and swapping. Paging will hold items that haven’t been accessed in a long time whereas swapping holds items that were recently taken out of memory. The items in pagingfile may not be accessed again for a long time whereas the items in swapfile might be accessed much sooner.

Conclusion

  1. “swapping” means put a whole program / process to swapspace on disk, while during “paging” only pages of memory are put onto disk (page = 4kb)
  2. because of its granularity, paging is often more efficient than swapping out the whoule process
  3. both mechanism can lead to performance degradation because content must be read out of multiple time slower disk storage instead of memory
  4. Windows 8 has both, pagefile and swapfile; the c:\swapfile.sys is used for the apps only
Advertisements

Windows Defragmentation Decision Process

Windows Server 2012 Defrag does not just defragment Volumes like in earlier versions. There’s a decision process behind, that selects the appropriate method for each volume.

Decision Process

The following commands are based on the new Optimize-Volume PowerShell cmdlet. Most of the parameters correspond to defrag.exe’s parameters. The Decision process works like this.

# For HDD, Fixed VHD, Storage Space:
Optimize-Volume -Analyze -Defrag

# Tiered Storage Space
Optimize-Volume -TierOptimize

# SSD with TRIM support
Optimize-Volume -Retrim

# Storage Space (Thinly provisioned), SAN Virtual Disk (Thinly provisioned), Dynamic VHD, Differencing VHD
Optimize-Volume -Analyze -SlabConsolidate -Retrim

# SSD without TRIM support, Removable FAT, Unknown
No operation.

Graphical Defrag Tool

The classical GUI Tools for Defrag still extists. If you open it, you’ll see theres a predefined schedule for a weekly defragmentation of your system volume. Depending on the type of storage you’re using, defrag only will run a short trim or other optimization at that time. In virtualized environments, you have either thin provisioned storage from vSphere or from storage. Because of this, Defrag will not start a classical defragmentation anymore on VM’s. Instead, a re-Trim / Slab-consolidation will start and takes only a few seconds / minutes to complete (depends on size).

PowerShell cmdlet

Server 2012 R2 also has a PowerShell cmdlet called “Optimize-Volume” that can be used instead of the classic defrag.exe tool. Both can handle the same functions, the cmdlet has an additional StorageTier Optimization function for Storage Spaces.

Information about the cmdlet is here:
http://technet.microsoft.com/en-us/library/hh848675.aspx

Get Pagefile usage of all running VM’s in inventory

Get Pagefile usage of all running VM’s in inventory

function Get-PagefileUsage {
$tabelle = @()
Get-VM | ? {$_.PowerState -like "PoweredOn"} | % { 
   $collect                =   "" | Select Name, PagefileUsage, TotalvRAM
   $collect.Name           =   $_.Name
   $collect.PagefileUsage  =   (Get-WmiObject Win32_PagefileUsage -ComputerName $_.Guest.Hostname).CurrentUsage
   $collect.TotalvRam      =   $_.MemoryMB
   $tabelle += $collect
   }
$tabelle | Export-Csv $env:userprofile\Desktop\PagefileOverview.csv
Write-Host -fore yellow "Tabelle exportiert auf den Desktop als PagefileOverview.csv"
""
}

SSD Performance Tests

We’ve been surprised that our very expensive Samsung SSD’s buyed some years ago for more CHF 800-1000 per 256GB of size (1+2) is not as fast as any other traditional Harddisk. Compared to actual newer ones from Samsung 470 series, I’m sure we were the ones that buyed for the technology research department of Samsung by buying the SSD’s for many bucks during their development.
By taking a look onto the newest model’s performance, it could be that the research was successful and the SSD’s we buy now are really performant ones.

(1) VBM2, Samsung 256GB, unknown model, black cover
(2) VBM1, Samsung 256GB, unknown model, silver cover
(3) AXM0, Samsung 470 SSD 128GB, shipped by Dell
(4) AXM0, Samsung 470 SSD 256GB, ordered in Switzerland
(5) 1.2, Corsair 128GB with Sandforce-2 Controller (uses compression)
(6) 391A, Patriot Wildfire 256GB black
(7) OCZ Vertex 2 128GB

Disk-ID 1 2 3 4
Seq-Read 156.56 MB/s 154.17 MB/s 243.48 MB/s 244.34 MB/s
Seq-Write 7.83 MB/s 151.14 MB/s 193.47 MB/s 235.91 MB/s
4K-Read 12.25 MB/s 12.27 MB/s 12.17 MB/s 13.29 MB/s
4K-Write 2.71 MB/s 5.82 MB/s 29.29 MB/s 42.89 MB/s
4K-64-Read 14.79 MB/s 15.95 MB/s 103.62 MB/s 106.99 MB/s
4K-64-Write 3.29 MB/s 4.98 MB/s 44.10 MB/s 47.56 MB/s
Acc.Time Read 0.47 ms 0.50 ms 0.28 ms 0.24 ms
Acc. Time Write 0.77 ms 0.61 ms 0.12 ms 0.11 ms
Score Read 43 44 140 145
Score Write 7 26 93 114
Total Score 70 97 307 337

Disk-ID 5 6 7
Seq-Read 194.71 MB/s 465.45 MB/s 200.87 MB/s
Seq-Write 79.17 MB/s 156.18 MB/s 82.94 MB/s
4K-Read 13.38 MB/s 10.10 MB/s 13.80 MB/s
4K-Write 10.07 MB/s 7.84 MB/s 24.16 MB/s
4K-64-Read 58.78 MB/s 111.76 MB/s 128.32 MB/s
4K-64-Write 38.69 MB/s 66.07 MB/s 75.99 MB/s
Acc.Time Read 0.19 ms 0.37 ms 0.24 ms
Acc. Time Write 0.36 ms 0.47 ms 0.33 ms
Score Read 92 168 162
Score Write 57 90 108
Total Score 195 332 353

Tests 1-6 made with a internal SATA Controller of Dell’s E6320 Notebook, AHCI enabled. For Test 7 I don’t have futher information about the controller and the configuration.

Some Notes to SQL and Performance

Some months ago, I tried to start a own Wiki site to maintain Know-How. Unfortunately I stopped filling it with updates as soon as I started this Blog 🙂

To not loose the small but interesting Knowledge, I want just put it in here:

RAID Layouts

Dell uses only a 2-Disk RAID1 LUN for Database Logfiles, while using a 10-Disk RAID10 for Database.

Performance Tests

There are three basic parameters to alter when running the I/O tests: block size, read/write, and serial/random. For SQL Server, the minimum tests to run are 8-KB and 64-KB block sizes, both serial and random, and both read and write. Most of the ratings you find on the Web are stated in throughput of bytes/sec for the 64-KB serial read test (which gives the highest throughput rating). A value of over 250 MB/sec per core for today’s computers is typically sufficient.

external Links

Scaling Up Your Data Warehouse with SQL Server 2008
Performance of SQL Server 2005 on ESX Server 3.5 

Benchmark Tools

SQLIO Disk Subsystem Benchmark Tool www.microsoft.com
IOmeter Project www.iometer.org

What does *CPU ready time* mean

In ESX / vSphere Environments you maybe already discovered a value called “CPU ready time”. And the confusing thing is, this value doesn’t tell you how long the VM was ready this day 😉

The whole definition can be found fully detailed in a VMware PDF here.

To get a quick explanation, I found a good forum thread where it’s explained quite good:

mcowger:
The rub comes when you have multi core VMs – regardless of the amount of CPU needed by the VM (lets say its a 4 core VM, and only needs 1 core worth of power), it needs to schedule onto 4 cores (because it is a 4 core VM).  While it is scheduled onto 4 processor cores, nothign else can use them, even though that VM isn’t really using all that they can do.  So, when you have a lot of low usage VM’s with 2 or 4 cores, you end up with problems scheduling them onto cores, and not so much the speed of the CPUs themselves.  Hence, the need to watch your CPU Ready times.

Source: http://communities.vmware.com/thread/130310

Internet Speed Test [updated]

Eine gute Internetverbindung braucht heute jeder. Mein Budget für den Internetzugang habe ich auf Fr. 50.- monatlich beschränkt. Als ich aber folgende Preis / Leistungsunterschiede sah, wollte ich die Angebote mal kurz vergleichen. Beide garantieren den Speed nur innerhalb der Schweiz. Da ich aber gerade von Cablecom die schlechten Übertragungsraten ins Ausland kenne habe ich einen Kollegen der bereits UPC Kunde mit der 100’000er Leitung für Fr. 75.-/Monta ist gebeten einen Test zu machen.

Im Test laden wir eine Debian ISO CD herunter. Dazu benutze ich den Firefox mit dem DownThemAll Plugin, welches den Download in vier Teile splittet, um möglichst mehr aus der Verbindung herauszuholen.

Quellen

Schweiz: mirror.switch.ch
Deutschland: ftp.informatik.uni-frankfurt.de
England: ftp.ticklers.org
USA 1: ftp.utexas.edu
USA 2: mirrors.xmission.com

ADSL Swisscom Business in Stans, Nidwalden

Quelle

Durchsatz min

Durchsatz max

Durchschnittlich

Schweiz

419 kb/s

537 kb/s

500 kb/s

Deutschland

479 kb/s

538 kb/s

500 kb/s

England

446 kb/s

544 kb/s

500 kb/s

USA 1

476 kb/s

534 kb/s

500 kb/s

USA 2

463 kb/s

537 kb/s

500 kb/s

10. Mai 2011 nach 10:00 Uhr CEST


VDSL Swisscom/Bluewin in Meggen, Luzern

Downlink: 5’000 kbit/s
Uplink: 500 kbit/s
monatlich Fr. 49.-

Quelle

Durchsatz min

Durchsatz max

Durchschnittlich

Schweiz

626 kb/s

666 kb/s

630 kb/s

Deutschland

566 kb/s

725 kb/s

640 kb/s

England

571 kb/s

664 kb/s

630 kb/s

USA 1

519 kb/s

770 kb/s

600 kb/s

USA 2

402 kb/s

750 kb/s

630 kb/s

29. Mai 2011 nach 17:30 Uhr CEST

Cablecom UPC beim Kollegen in Würzenbach, Luzern

Downlink: 25’000 kbit/s
Uplink: 2’500 kbit/s
montalich Fr. 50.-

Quelle

Durchsatz min

Durchsatz max

Durchschnittlich

Schweiz

3295 kb/s

9528 kb/s

6718 kb/s

Deutschland

1356 kb/s

6875 kb/s

3792 kb/s

England

730 kb/s

8580 ks/s

5508 kb/s

USA 1

13 kb/s

1526 kb/s

479 kb/s

USA 2

17 kb/s

2480 kb/s

1175 kb/s

30. Mai 2011 nach 17:00 Uhr CEST

VDSL Green.ch in Meggen, Luzern
Downlink: 40’000 kbit/s
Uplink: 4’000 kbit/s
monatlich Fr. 59.-

Quelle

Durchsatz min

Durchsatz max

Durchschnittlich

Schweiz

2560 kb/s

3320 kb/s

2614 kb/s

Deutschland

2150 kb/s

2920 kb/s

2560 kb/s

England

1020 kb/s

1730 kb/s

1371 kb/s

USA 1

158 kb/s

505 kb/s

279 kb/s

USA 2

513 kb/s

839 kb/s

610 kb/s

Den Durschnittswert habe ich mit einer Messung über zwei Minuten berechnet.

01. Juni 2015 nach 18:50 Uhr CEST

VDSL Green.ch in Meggen, Luzern
Downlink: 100’000 kbit/s
Uplink: 10’000 kbit/s
monatlich Fr. 59.-

Quelle

Durchsatz min

Durchsatz max

Durchschnittlich

Schweiz

7200 kb/s

8570 kb/s

7500 kb/s

Deutschland*

5950 kb/s 12010 kb/s 9133 kb/s

England

2850 kb/s 10070 kb/s 7633 kb/s

USA 1

3580 kb/s 11020 kb/s 7166 kb/s

USA 2

3240 kb/s 8380 kb/s 6316 kb/s

Den Durschnittswert habe ich mit einer Messung über zwei Minuten berechnet.
Angepasste Quellen: Deutschland: ftp.tu-chemnitz.de

13. Juni 2015 nach 08:30 Uhr CEST
×