AMBER Archive (2007)

Subject: RE: AMBER: amber on AMD opteron-250

From: Ross Walker (ross_at_rosswalker.co.uk)
Date: Tue Dec 11 2007 - 15:20:11 CST

Hi Servaas,

> I mean that we have another cluster with intel processors and there
> there is no problem if I use more then 4 processors. The
> communicaction
> there is with a 100Mbps ethernet

This is very interesting. That suggests to me one of two things, assuming it
is definately using the 100Mbps ethernet for MPI traffic and doesn't have
something like Scali hidden from the user.

1) The processors are much much slower than the AMD chips and there are less
per node. I.e. single cpu 2GHz P4 chips.

2) Alternatively or in addition this machine has very good switches and very
good network cards installed likely with the TCP/IP buffers set well on the
machine and with flow control turned on.

The key is that it is not bandwidth that is limiting AMBER but latency.
Without flow control the latency as a function of bandwidth load is
disasterous. Everything looks fine until you get to say 90% utilization -
which an all to all MPI call will do - and then your latency goes from one
millisecond to 2 seconds! due to packet loss.

So I would take a look at how they have setup this machine.

All the best
Ross

/\
\/
|\oss Walker

| HPC Consultant and Staff Scientist |
| San Diego Supercomputer Center |
| Tel: +1 858 822 0854 | EMail:- ross_at_rosswalker.co.uk |
| http://www.rosswalker.co.uk | PGP Key available on request |

Note: Electronic Mail is not secure, has no guarantee of delivery, may not
be read every day, and should not be used for urgent or sensitive issues.

-----------------------------------------------------------------------
The AMBER Mail Reflector
To post, send mail to amber_at_scripps.edu
To unsubscribe, send "unsubscribe amber" to majordomo_at_scripps.edu