Log Likelihood - Statistics Toolbox

I am using the Statistics Toolbox DFITTOOL to find different pdf's to
a data set. The results produce a log likelihood ratio value and
estimates of covariance etc. Aside from just seeing visually which
fit is better, is it true that the closer the log likelihood is to
unity the better??? [Although a few fits look good, the numbers I
obtain are of the order of -ve millions - does this make sense?]

Hi Rob,

What do you mean by 'the closer the log likelihood is to unity the better'?

I guess you are asking how you should use these log likelihood values. To
answer your question, I cite a definition of likelihood as follows.

"Likelihood is the hypothetical probability that an event that has already
occurred would yield a specific outcome. The concept differs from that of a
probability in that a probability refers to the occurrence of future events,
while a likelihood refers to past events with known outcomes."

Hope this helps.
Bin


"Rob Kaw" <k76_rb@nospam.no.org> wrote in message
news:ef125e2.-1@webx.raydaftYaTP...
>I am using the Statistics Toolbox DFITTOOL to find different pdf's to
> a data set. The results produce a log likelihood ratio value and
> estimates of covariance etc. Aside from just seeing visually which
> fit is better, is it true that the closer the log likelihood is to
> unity the better??? [Although a few fits look good, the numbers I
> obtain are of the order of -ve millions - does this make sense?]

>>I am using the Statistics Toolbox DFITTOOL to find different pdf's to 

Adding to what Bin wrote, it's often hard to interpret raw likelihood
values.  For discrete data where the likelihood is the product of
probabilities, 1 would be an ideal case that is not usually achievable.  (I
assume meant to compare the likelihood, not the log likelihood, to 1.)  For
continuous data the likelihood is the product of density values and could be
any positive number.

People compare two models using their likelihood ratio, or the difference in
their log likelihoods.  The theory for this is developed best when one model
is a special case of the other.  For example, you could compare an
exponential fit to either a gamma or a Weibull fit using a likelihood ratio.

You sometimes see things like AIC (Akaike's information criterion) defined
as functions of a likelihood.  Even for AIC, I think, it's the difference in
AIC values between two models, not the absolute values, that are important.

-- Tom

Hi, thanks for the information. I mean when I generated a fit to data
using the distribution fitting tool, the log liklihood ratio is
shown. So for example, using pdf A the log liklihood might be
-3.1E+006, and with B it might be -3.4E+006; and both look roughly
similar, so I wondered if I can say which is better based on the LL
figures??? Thanks.

Tom Lane wrote:
>
> 
> pdf's to 
value and 
visually
> which 
likelihood is
> to 
numbers
> I 
sense?]
>
> Adding to what Bin wrote, it's often hard to interpret raw
> likelihood
> values. For discrete data where the likelihood is the product of
> probabilities, 1 would be an ideal case that is not usually
> achievable. (I
> assume meant to compare the likelihood, not the log likelihood, to
> 1.) For
> continuous data the likelihood is the product of density values and
> could be
> any positive number.
>
> People compare two models using their likelihood ratio, or the
> difference in
> their log likelihoods. The theory for this is developed best when
> one model
> is a special case of the other. For example, you could compare an
> exponential fit to either a gamma or a Weibull fit using a
> likelihood ratio.
>
> You sometimes see things like AIC (Akaike's information criterion)
> defined
> as functions of a likelihood. Even for AIC, I think, it's the
> difference in
> AIC values between two models, not the absolute values, that are
> important.
>
> -- Tom
>
>
>

> Hi, thanks for the information. I mean when I generated a fit to data
> using the distribution fitting tool, the log liklihood ratio is
> shown. So for example, using pdf A the log liklihood might be
> -3.1E+006, and with B it might be -3.4E+006; and both look roughly
> similar, so I wondered if I can say which is better based on the LL
> figures??? Thanks.

Rob, if one fit is a special case of the other you can do a likelihood ratio
test.  For information see any statistics book or

http://en.wikipedia.org/wiki/Likelihood-ratio_test

Twice the log of the ratio of the larger likelihood to the smaller (twice
the difference in log likelihoods) can be used as a test statistic having a
chi-square distribution if the true distribution is the more constrained
fit.  For example, the exponential is the same as a Weibull with the shape
parameter constrained to be 1.  If the true distribution is exponential, the
likelihood ratio comparing this to a Weibull fit will have a chi-square
distribution with 1 d.f. because there is one constraint.

I'm not aware of any theory allowing you to use the likelihoods this way to
compare unrelated distributions with a formal test.

-- Tom