Subject: Re: Parallel ALS-WR on very large matrix -- crashing (I think)


Nicholas,

can you give us the detailed arguments you start the job with? I'd
especially be interested in the number of features (--numFeatures) you
use. Do you use the job with implicit feedback data
(--implicitFeedback=true)?

The memory requirements of the job are the following:

In each iteration either the item-features matrix (items x features) or
the user-features matrix (users x features) is loaded into the memory of
each mapper. Then the original user-item matrix (or its transpose) is
read row-wise by the mappers and they recompute the features via
AlternatingLeastSquaresSolver/ImplicitFeedbackAlternatingLeastSquaresSolver.

--sebastian
On 02.02.2012 09:53, Sean Owen wrote: