AMBER Archive (2009)

Subject: Re: [AMBER] ptraj limit on trajectory size

From: Sasha Buzko (obuzko_at_ucla.edu)
Date: Mon Aug 24 2009 - 17:31:14 CDT

Thanks, Ross.
Yes, increasing the limit certainly looks like WORK. I'll just leave it
alone and do clustering in chunks of the total trajectory.

Sasha

Ross Walker wrote:
> Hi Sasha,
>
> Looking at the code in clusterLib.c one can find the source of the warning:
>
> CLUSTER_API SymmetricMatrix* AllocateSymmetricMatrix(int Size)
> {
> SymmetricMatrix* pMatrix;
> /* */
> if (Size >= 32768) {
> fprintf(stderr, "Error in AllocateSymmetricMatrix, Size (%d) of
> SymmetricMatrix is too big. Please limit the size to be less
> than 32768.\n", Size);
> exit(1);
> }
> unsigned int DataBlockSizeInBytes;
> pMatrix = (SymmetricMatrix*)SafeMalloc(__FILE__, __LINE__,
> sizeof(Matrix));
> pMatrix->Size = Size;
> DataBlockSizeInBytes = (sizeof(float) * Size * (Size+1)) / 2;
> pMatrix->Data = (float*)SafeMalloc(__FILE__, __LINE__,
> DataBlockSizeInBytes);
> memset(pMatrix->Data,0,DataBlockSizeInBytes);
> return pMatrix;
> }
>
> The issue at hand is the size of DataBlockSizeInBytes. If you calculate this
> with size >= 32768 you find that it exceeds 2GB in size and this is the
> origin of the limit. Note in principle since DataBlockSizeInBytes is
> unsigned this could be increased to 4GB which would be 46341. However, this
> might break the code elsewhere if this value ends up in a signed int. In
> principle one could try recompiling with 64 bit ints which if the unsigned
> works then the limit could be 3037000500 assuming of course that you have 16
> exa bytes of memory.
>
> I have no idea though if a 64 bit integer compile will break anything else
> in the code. Often problems occur if integer offsets are assumed or it links
> to some external library that expects 32 bit ints. I think ptraj does not do
> this though so you could try forcing a 64 bit int compile and see if it 1)
> does not segfault and 2) passes some test cases. Obviously commenting out
> the check above.
>
> This of course does not help you with the huge memory requirement of
> clustering. Probably the algorithm should be updated to not require N^2
> memory if such memory is not available. This is unfortunately WORK. :-(.
>
> Good luck,
> Ross
>
> /\
> \/
> |\oss Walker
>
> | Assistant Research Professor |
> | San Diego Supercomputer Center |
> | Tel: +1 858 822 0854 | EMail:- ross_at_rosswalker.co.uk |
> | http://www.rosswalker.co.uk | PGP Key available on request |
>
> Note: Electronic Mail is not secure, has no guarantee of delivery, may not
> be read every day, and should not be used for urgent or sensitive issues.
>
>
>
>
>> -----Original Message-----
>> From: amber-bounces_at_ambermd.org [mailto:amber-bounces_at_ambermd.org] On
>> Behalf Of Sasha Buzko
>> Sent: Monday, August 24, 2009 10:00 AM
>> To: AMBER Mailing List
>> Subject: [AMBER] ptraj limit on trajectory size
>>
>> Hi all,
>> ptraj appears to have a hard-coded limit on the number of trajectory
>> frames that can be used as the input for clustering. I have a rather
>> long trajectory, and I keep hitting an AllocateSymmetricMatrix error
>> with a complaint that the size of the matrix should be less than 32768.
>> Is there a way to change this value in the source code and recompile
>> ptraj to handle bigger frame sets? Or is it an inherent algorithmic
>> restriction?
>>
>> Thanks for any suggestions
>>
>> Sasha
>>
>> _______________________________________________
>> AMBER mailing list
>> AMBER_at_ambermd.org
>> http://lists.ambermd.org/mailman/listinfo/amber
>>
>
>
> _______________________________________________
> AMBER mailing list
> AMBER_at_ambermd.org
> http://lists.ambermd.org/mailman/listinfo/amber
>
>
_______________________________________________
AMBER mailing list
AMBER_at_ambermd.org
http://lists.ambermd.org/mailman/listinfo/amber