Well, I can't say there was a consensus on this, but the replies were
informative. I called Sun Technical support as well and they said they
didn't see any reason to split swap any further than across multiple
SCSI controllers, although they did not provide any of the detailed
technical reasons the people on this list did. For the moment I've
decided to not spread the swap out across all the disks, but to
leave small unused partitions on each so I can do this in the future
if I want.
I forgot to save the headers from my original post, so this SUMMARY
Subject line may differ a bit from the original. The responses are below.
Thanks to:
mrs@cadem.mc.xerox.com (Mike Salehi)
Chris Terry/Computervision <Chris_Terry/Computervision.CVAUS@ausns1.cv.com>
Glenn.Satchell@Uniq.com.au (Glenn Satchell - Uniq Professional Services)
Ake Sandgren <ake@cs.umu.se>
martin@gea.hsr.it (Martin Achilli)
Jeff Victor <victoj@Sage.EDU>
as well as a couple of other unamed folks who's mail I accidently deleted.
/=/=/=/=/=/=/=/=/=/=/=/=/=/=/=/=/=/=/=/=/=/=/=/=/=/=/=/=/=/=/=/=/=/=/=/=/=/
Jim Napier jnapier@soe.ucsd.edu
Systems Administration (619)534-5212
School of Engineering
UC San Diego
/=/=/=/=/=/=/=/=/=/=/=/=/=/=/=/=/=/=/=/=/=/=/=/=/=/=/=/=/=/=/=/=/=/=/=/=/=/
My original post-
>I'm adding more disk to my system and am considering how to arrange
>extra swap space. I know that it's a good idea to divide swap up
>so that you have swap areas on different SCSI controllers. But I also
>have multiple disks on each controller and I was wondering if there's
>any reason to further interleave the swap across the disks that are
>on each controller. To be explicit let's say I have 3 SCSI controllers
>each with 2 2Gb disks on them. If I allocate 600Mb of swap, I'd want
>to put 200Mb on each controller. But is it worth it to then break that
>down so that each disk has 100Mb of swap? It's worth pointing out that
>none of these disks will be so heavily used that having swap and UFS
>file systems on the same disk is a problem. There are 2 apparently
>opposing arguments that I see as pertinent-
>
>1. Intelligent controllers in the disks can buffer data and do overlapped
> seeks so that it's possible I'll see a small improvement in data
> transfers by dividing up the swap more finely.
>
>2. Splitting the swap too finely may contribute nothing (or reduce
> efficiency) because I'm not taking advantage of reading multiple
> sectors of data sequentially off of a single disk.
>
>Both these arguments involve unknowns (at least to me) in how the
>controller firmware operates and how the kernel optimizes block
>allocations for efficient use of swap.
>
>Am I just overanalyzing what is a non-issue? Thanks for any help.
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Jim,
You have an intresting argument and if you come to any final
conclusion, let me know. But last I knew, it was a good idea to
do it as the seek operation was very slow compared to transfer rate, so
having 3x arms do the swap was more efficient.
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Mike (Mehran) Salehi mrs@cadem.mc.xerox.com (716) 422-2725
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Jim,
Reading your email it became obvious that you wouldn't receive the benefits you
expected unless at least two conditions became true on your system :-
1. A 1Mb or greater transfer from "real" memory into "anonymous" memory (read
swap) occours. This forces the selection of another swap area other than the
primary from the available swap device "pool".
2. You tune the maxpgio value in the kernel paramter file (/etc/system) to suit
the number and speed of the drives that you will be swapping on. If you have
5400 rpm disks, then maxpgio should be set to 60 * n drives. If you have 7200
rpm disks, then maxpgio should be set to 90 * n drives. This "instructs" or
provides a hint to the kernel that a far greater rate of paging can occour than
the standard (a single drive @ 60 pages/sec)
ie set maxpgio=240 # this is four four swap devices @ 60 pages/sec
(this info is in Adrian Cockcrofts book by the way)
Hope this is useful for you
Chris
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
I think three swap areas would be enough. I just did a 'Concepts and
Tuning' course a few weeks back at the local Sun office - swap is
interleaved in 1MB chunks. This means it is allocated from one area till
approximately 1 MB is used, then switches to the next and so on in a
round robin. Having too many swap areas isn't going to make much
difference.
regards,
-- Glenn Satchell glenn@uniq.com.au | There's a fine line Uniq Professional Services Pty Ltd ACN 056 279 335 | between fishing and PO Box 70, Paddington, NSW 2021, (Sydney) Australia | standing on the shore Phone 02 380 6360 Pager 016 287 000 Fax 02 380 6416 | looking like an idiot.++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
You'd be amazed at how much better a system can work with multiple diskarms for swap. Going from1 to 2 diskarms is a MAJOR win for ANY system. (In my experience anyway). There is also another thing to consider. If you ever want to catch a system coredump you will need the first swap area to be >= Prim.memory size (256M in your case) the dumps go ONLY to this partition.
Swap is allocated in !MB chunks so a 4MB swapallocation is spread out in round robin fashion over each swaparm. And since it uses 1MB blocks you get all the benefits of the controller/disk per 1MB block (it's big enough) this means that for each diskarm you can use you will get better swap performance (at least up to some, for me, unknown limit)
And one more thing, SunOS5.4 uses the swap partition(s) a lot more then you would guess from start.
So, my conclusion is, use 6 swaparms if you have it!!! If having linear behaviour all the way up to 600M USED swap then put 600/x on each x arms. If you want to catch system dumps put 256M on first arm and spread the reast equally.
Good luck! -- Ake Sandgren, Dep. of Computing Science, Umea University, S-90187 Umea, Sweden Internet: ake@cs.umu.se Phone: +46 90 166134 Fax: +46 90 166126 WWW: http://www.cs.umu.se/~ake (PGP key is found here)
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
I think you should split your swap across controllers, splitting across the disks for each controller is a bit overkill I think. Some of the new HDs can use all of a controller's bandwidth (like barracudas) so I think you won't gain much by splitting across each disk for a given controller. It also depends on how much your system is swapping, you can observe this with vmstat. Iostat will also give you the average disk usage (including swap activity I think) so check if disk usage is balanced. We have two server machines with 4 disks each (on one controller though), with iostat I discovered that disk usage was unbalanced and moved some stuff so as to balance it. Don't waste too much time on this, get faster disks if you want performance.
Martin
Martin Achilli - martin@gea.hsr.it Consiglio Nazionale delle Ricerche - I.N.B. c/o Medicina Nucleare, Osp. San Raffaele 20132 Milano, Italy tel: +39/2/26433648 fax: ../26415202
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
On Wed, 21 Feb 1996, Jim Napier wrote:
> 1. Intelligent controllers in the disks can buffer data and do overlapped > seeks so that it's possible I'll see a small improvement in data > transfers by dividing up the swap more finely. > > 2. Splitting the swap too finely may contribute nothing (or reduce > efficiency) because I'm not taking advantage of reading multiple > sectors of data sequentially off of a single disk. >
I think that (1) is a more important factor than (2).
> Am I just overanalyzing what is a non-issue? Thanks for any help.
If you aren't seeing any performance problems, then don't try too hard. If the server is put under a much a greater load later, you should try to find where the bottleneck *is*, before making "stabs in the dark". If swapping is causing a problem, then you can fine tune the swap space layout. But if it's not, then you've wasted your time.
------------------------------------------------------------------------------ Jeff Victor The SageNet `Net Sage' victoj@sage.edu Network Administrator 518-270-2354 The Sage Colleges Any opinions detected here are Troy, New York probably your imagination running wild. -------------------------------------------------------------------------------
This archive was generated by hypermail 2.1.2 : Fri Sep 28 2001 - 23:10:54 CDT