SUMMARY: Private ethernet between servers?

From: Charles Butcher (charlesb@nimrod.ta.oz.au)
Date: Mon Jun 28 1993 - 07:23:17 CDT


Here is the original question:

> It seems that any time you buy an Sbus card these days it also has an
> ethernet controller on it; a sort of "gift with purchase" if you like.
> Right now I have a 670 with 2 ports and a 670 with FIVE!
>
> It seems a shame to let these things sit idle, so I had the idea of
> setting up a private ethernet for the two servers, so they could
> chatter away to each other on what would be almost a point-to-point
> 10Mb link (only 2 boxes on the wire...). This would keep a resonable
> chunk of NFS traffic and all the rdump traffic off the main wire.
>
> I have two questions regarding this:
>
> 1) How would I fix things so that nothing other than
> server-to-server traffic gets routed over this link?
>
> Note that at least one of the servers will have 3 ethernets
> connected, and so will probably need to run routed. The other one
> could conceivably route statically if it would make things easier.

> 2) Is it possible to use an AUI cable (with appropriate cross-overs)
> to connect the server's AUI ports together directly (the ethernet
> equivalent of a null modem cable, I suppose)? Alternatively, could
> they be wired together directly using a 10base-T cable (again with
> crossovers if necessary)?

I will deal with Question 2 first because the answer is straightforward:

QUESTION 2
----------

- Don't connect AUI ports together -- this wasn't meant to happen.

- It is perfectly acceptable to connect 10Base-T ports together; in fact
  perryh@pluto.rain.com (Perry Hutchison) mentions that the 10Base-T
  specification explicitly allows for direct connection of exactly two
  devices. Several people supplied diagrams for the cross-over cable
  needed; thanks to mandrews@vcrt.hq.videocart.com (Mike Andrews) for
  the following:

  RJ-45 Pin Signal Signal RJ-45 Pin

        1 TX+ ---------- RX+ 3
        2 TX- ---------- RX- 6
        3 RX+ ---------- TX+ 1
        6 RX- ---------- TX- 2

QUESTION 1
----------

As I understand it, the most practical approach to question 1 would be as follows:

1) Allocate a new network number for the network and the two interfaces. Don't
   be tempted to re-use another network number. If the private link ever escapes
   you'll have lots of routing fun. -- gordonr@nms.otc.com.au (Gordon Rowell)

2) Each server should now have a new IP address for its private
   interface. Give this a descriptive name such as hostname-A or
   hostname-le1. Use this name in NFS mounts, rdumps telnet sessions
   etc., whenever you want a connection to go via the private link.

   - It was suggested to me by walter@aus.sun.com (Walter Schroeder) that on such
     a link you may as well increase the size of your NFS read & write buffers
     to 8k. You do this via the NFS-specific mount options: rsize and wsize.

3) Use static routing on the servers if possible. In many cases it will be.
   Here is a practical example from John A. Murphy <jam@philabs.philips.com>:

> We have 5 servers on our computer room network (which is in reality
> just a Cabletron MTU). Each machine on this subnet has a set of routes
> to each host. In the host table, each host has 2 names, name and
> name-cr. So we have routes that read
>
> for h in wizard condor betty misty
> do
> echo -n "$h..."
> route add host $h ${h}-cr 1
> done
>
> We also have static routes for the other subnets that point to the
> router so only the individual host traffic goes to this subnet.
>
> For these routes, netstat -r looks like...
>
> Routing tables
> Destination Gateway Flags Refcnt Use Interface
> betty betty-cr UGH 1 157170 le1
> condor condor-cr UGH 2 777986 le1
> wizard wizard-cr UGH 3 1692316 le1
> misty misty-cr UGH 0 128251 le1

And that should be all you need to do.
The next two points are for those who have more complex routing requirements...

4) If you want to run routed you can use the -q flag to stop it from
   passing on routing information. You can also give the "private" (as
   opposed to "-private" flag to ifconfig(8) in /etc/rc.local. You can
   also disable IP forwarding in the kernel by putting
           'options IPFORWARDING=-1'
   in the kernel config file.
   
   Some folks suggested using gated instead of routed in situations of greater
   routing complexity.

5) For those using DNS the following may be of interest:

> From: Hans van Staveren <sater@cs.vu.nl>
>
> If you use DNS we use a special patch to use preferred networks.
> Replace in.named by our special one and run 'in.named -i le3' where le3
> is your preferred network. If a host has an address on the network
> connected to that interface it will be used. We use the same trick
> for our mostly parallel FDDI network.
 
 
Guest Stars (in order of appearance):
-------------------------------------
"Rogerio Rocha - DI - BVL" <rogerio@bvl.pt>
<Piete.Brooks@cl.cam.ac.uk>
<gdmr@dcs.ed.ac.uk>
Tim Beyea <beyea@ERC.MsState.Edu>
Hans van Staveren <sater@cs.vu.nl>
mharris@jpmorgan.com (Michael Harris)
Steve Lodin <swlodin@kocrsv01.delcoelect.com>
ericb@telecnnct.com (Eric Burger)
Mike Raffety <miker@il.us.swissbank.com>
hyder@niwot.scd.ucar.EDU (Paul Hyder)
mandrews@vcrt.hq.videocart.com (Mike Andrews)
rjcronin@uop.com (Robert J. Cronin)
John A. Murphy <jam@philabs.philips.com>
Barry Margolin <barmar@Think.COM>
perryh@pluto.rain.com (Perry Hutchison)
ivan@durras.anu.edu.au (Ivan Angus - ne Dean)
gordonr@nms.otc.com.au (Gordon Rowell)
walter@aus.sun.com (Walter Schroeder) -- face-to-face conversation

        Thanks Folks!

--
Charles Butcher           | Remember that a computer which is experiencing
charlesb@nimrod.ta.oz.au  | an electrical fire is not very secure.



This archive was generated by hypermail 2.1.2 : Fri Sep 28 2001 - 23:07:58 CDT