Re: Zero-Inflated Models, using NLMIXED is it really bizarre,

Thank you very David for your valuable explanations.

1-)I would just ask you how to get the clue to
looking into using the PARMS statement with an input
data set of possible start values, so that the system can look
over more than one local optimum. Is there a options in PARMS statement which helps to find the maximum optimum?

2-) You are asking me: " Do your data really warrant the use of such models? "
In my post, I'v sent my real data. What do think about it ?

Again thanks a lot.
Hamani.

-----Original Message-----
From: SAS(r) Discussion [mailto:SAS-L@LISTSERV.UGA.EDU]
Sent: November 21, 2006 1:29 AM
To: SAS-L@LISTSERV.UGA.EDU
Subject: Re: For Zero-Inflated Models, using NLMIXED is it really
bizarre, so not at all useful.


Hamani.Elmaache1@CRA-ARC.GC.CA wrote:
>Hi every body.
>
> I'm trying to get parameter estimates from population. For that, I'm
>using different Zero-Inflated Models. What I noticed is
>1-) When I change values in "parms" I get completely different parameter
>estimates.
>2-) When I change distributions and values in "parms" I also get
>completely different parameter estimates.
>3-) Worse than that, the predicted values are completely different from
>dependent variable values.
>4-) Although, all these 5 models satisfy the convergence criterion, they
>ARE NOT consistent.
>
>What the use of doing Zero-Inflated Models, if they don't produce any
>results?
>
>Here below the data and 5 codes of these Zero-Inflated Models if someone
>wants give them a look.
>
>Thanks in advance for any help or comments.
>
>/********************************/
>
>data test;
>input Dollar CriterionA CriterionC CriterionD CriterionE;
>datalines;
> 768.20 13.1 100 9.5
>100
> 0.00 1.91 100 1.9
>0
> 0.00 4.4 100 3.18
>0
> 700.75 16.06 100 13.68
>100
> 348.91 40.92 0 40.92
>0
> 0.00 2.87 100 2.9
>100
> 1321.28 12.53 100 20.38
>100
> 0.00 34.16 0 34.16
>0
> 514.78 13.25 100 5.74
>100
> 1008.91 9.5 100 12.54
>100
> 183.91 1.87 100 1.86
>0
> 0.00 12.5 100 4.02
>0
> 184.91 2.29 100 2.28
>100
> 121.76 3.96 100 2.86
>100
> 729.95 19.8 100 17.46
>0
> 268.76 6.42 100 6.42
>100
> 0.00 7.37 100 5.32
>100
> 0.00 9.65 0 9.66
>0
> 0.00 2.47 100 2.9
>100
> 252.50 3.96 100 2.86
>100
> 206.16 4.94 100 4.92
>100
> 0.00 2.24 100 0
>100
> 55.56 1.82 100 1.3
>100
> 0.00 8.18 100 5.92
>100
> 197.47 4.73 100 4.7
>100
> 0.00 14.08 100 14.08
>100
> 0.00 4.37 100 3.16
>100
> 0.00 8.1 100 5.88
>100
> 2848.56 65.33 100 68.22
>100
> 63.29 1.43 100 1.22
>0
> 227.02 3.3 100 1.12
>0
> 75.48 2.49 100 1.78
>100
> 0.00 2.29 100 1.72
>0
> 0.00 7.39 100 5.4
>100
> 1269.42 15.94 100 15.94
>100
> 955.45 17.6 100 14.84
>100
> 98.55 50.76 100 1.36
>100
> 0.00 119.61 0 157.68
>0
> 622.84 52.8 100 52.8
>0
> 78.19 51.72 100 31.88
>100
> 2498.42 22 100 21.98
>100
> 0.00 146.63 0 164.38
>100
> 1.46 73.29 100 86.6
>0
> 842.52 14.65 100 11.56
>0
> 912.79 28.54 100 11
>0
> 451.51 6.34 100 6.36
>100
> 0.00 41.44 0 30.16
>0
> 670.03 12.19 100 8.88
>0
> 107.48 1.87 100 1.36
>100
> 331.23 4.4 100 4.38
>100
> 434.77 4.4 100 4.4
>100
> 934.65 13.2 100 13.18
>100
> 0.00 1.5 100 1.5
>100
> 1660.17 30.59 100 22.24
>100
> 553.84 7.39 100 7.36
>100
> 1686.40 28.54 100 20.74
>0
> 369.91 12.15 100 8.84
>100
> 0.00 1.24 100 1.46
>0
> 382.42 7.08 0 5.16
>0
> 0.00 2.6 100 3.44
>100
> 0.00 97.35 0 128.3
>0
> 0.00 11.24 100 8.16
>100
> 0.00 54.94 0 30.42
>0
> 0.00 44 0 57.98
>0
> 1122.59 15.83 100 15.84
>100
> 0.00 24.46 100 18.98
>100
> 957.79 16.12 100 11.7
>100
> 0.00 34.67 0 25.2
>0
> 3215.60 44 100 43.98
>100
> 4510.93 54.25 100 60.26
>100
> 144.28 2.46 100 1.8
>0
> 0.00 1.87 100 1.7
>100
> 489.44 4.4 100 4.4
>0
> 1260.68 16.94 0 15.96
>0
> 1749.92 28.54 100 20.72
>0
> 168.17 6.76 100 4.9
>0
> 1885.40 66.26 100 37.7
>100
> 0.00 158.4 0 94
>100
> 568.11 7.48 100 6.96
>100
> 0.00 8.8 100 10.36
>100
> 687.55 9.68 100 9.68
>100
> 6192.33 85.1 100 85.12
>100
> 913.21 15.45 100 11.22
>100
> 56.15 7.92 100 1.12
>100
> 0.00 46.81 100 34.1
>100
> 0.00 111.86 100 108.38
>100
> 1681.89 23.1 100 23.1
>100
> 0.00 30.8 100 22.4
>100
> 59.59 1.6 100 1.16
>100
> 0.00 10.77 100 7.84
>100
> 4108.56 28.12 100 20.44
>100
> 0.00 26.4 0 31.18
>0
> 772.35 39.6 100 10.54
>0
> 983.79 16.65 100 12.1
>100
> -285.85 18.63 100 13.52
>100
> 5758.31 61.78 100 73.86
>100
> 361.61 9.29 100 6.74
>100
> 0.00 30.8 100 22.38
>100
> ;
> run;
>
> data no_zero;
> set test;
> if Dollar ne 0;
> run;
>
> proc gchart data=test;
>vbar Dollar;
>run;
>quit;
> proc gchart data=no_zero;
>vbar Dollar;
>run;
>quit;
>
>/********************************/
>/********************************/
>proc nlmixed data=test MAXITER=10000 ;
>parms p0= 0.36
>A0 = 0.5 0.6 0.7 0.8 0.9 1 1.1 1.2 1.3
>RA =0.8 0.9 1 1.1 1.2 1.3
>RC =1.014
>RD =0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8
>RE = 5.607;
> mu=A0+RA*CriterionA +RC*CriterionC+RD*CriterionD+RE*CriterionE ;
>if Dollar=0 then prob = p0 +(1-p0)*pdf('gamma',Dollar,mu);
>else prob = (1-p0)*pdf('gamma',Dollar,mu);
> if (prob > 1e-8) then loglike = log(prob);
> else loglike = -1e19;
>model Dollar~ general(loglike);
>ods output ParameterEstimates=para
> (where=(Parameter not in( 'p0' 'A0' ) ));
>PREDICT mu OUT=PREDI_gam;
>run;
>/********************************/
>/********************************/
>proc nlmixed data=test MAXITER=10000 ;
>parms p0= 0.36
>A0 = 0.5 0.6 0.7 0.8 0.9 1 1.1 1.2 1.3
>RA =0.8 0.9 1 1.1 1.2 1.3
>RC =1.014
>RD =0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8
>RE = 5.607;
> mu=A0+RA*CriterionA +RC*CriterionC+RD*CriterionD+RE*CriterionE ;
>if Dollar=0 then prob = p0 +(1-p0)*pdf('gamma',Dollar,mu);
>else prob = (1-p0)*pdf('gamma',Dollar,mu);
> if (prob > 1e-8) then loglike = log(prob);
> else loglike = -1e19;
>model Dollar~ general(loglike);
>ods output ParameterEstimates=para
> (where=(Parameter not in( 'p0' 'A0' ) ));
>PREDICT mu OUT=PREDI_gam;
>run;
>
>/********************************/
>/********************************/
>
>proc nlmixed data=test MAXITER=10000 ;
>parms p0= 0.36
>A0 = 635
>RC =1.2
>RE =1.1 ;
> mu=A0 +RC*CriterionC +RE*CriterionE ;
>if Dollar=0 then prob = p0 +(1-p0)*pdf('gamma',Dollar,mu);
>else prob = (1-p0)*pdf('gamma',Dollar,mu);
> if (prob > 1e-8) then loglike = log(prob);
> else loglike = -1e19;
>model Dollar~ general(loglike);
>ods output ParameterEstimates=para
> (where=(Parameter not in( 'p0' 'A0' ) ));
>PREDICT mu OUT=PREDI_exp;
>run;
>/********************************/
>/********************************/
>proc nlmixed data=test MAXITER=10000 ;
>parms p0= 0.36
>A0 = 1500
>RA =0.8 0.9 1 1.1 1.2 1.3
>RC =1.014
>RD =0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8
>RE = 5.607;
> mu=A0+RA*CriterionA +RC*CriterionC+RD*CriterionD+RE*CriterionE ;
>if Dollar=0 then prob = p0 +(1-p0)*pdf('EXPONENTIAL',Dollar,mu);
>else prob = (1-p0)*pdf('EXPONENTIAL',Dollar,mu);
> if (prob > 1e-8) then loglike = log(prob);
> else loglike = -1e19;
>model Dollar~ general(loglike);
>ods output ParameterEstimates=para
> (where=(Parameter not in( 'p0' 'A0' ) ));
>PREDICT mu OUT=PREDI_exp;
>run;
>
>/********************************/
>/********************************/
>
>proc nlmixed data=test MAXITER=10000 ;
>parms p0= 0.36
>A0 = 2400
>RA =0.8 0.9 1 1.1 1.2 1.3
>RC =1.014
>RD =0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8
>RE = 5.607;
> mu=A0+RA*CriterionA +RC*CriterionC+RD*CriterionD+RE*CriterionE ;
>if Dollar=0 then prob = p0 +(1-p0)*pdf('EXPONENTIAL',Dollar,mu);
>else prob = (1-p0)*pdf('EXPONENTIAL',Dollar,mu);
> if (prob > 1e-8) then loglike = log(prob);
> else loglike = -1e19;
>model Dollar~ general(loglike);
>ods output ParameterEstimates=para
> (where=(Parameter not in( 'p0' 'A0' ) ));
>PREDICT mu OUT=PREDI_exp;
>run;

Okay. Let me take your main points one at a time.

>1-) When I change values in "parms" I get completely different parameter
>estimates.

This is a common problem with models so complex that they
have multiple local optima. You have lots of local 'bumps' in
your model surface, and any of these bumps can be a local
optimum, even if they are not your global optimum. You
should look into using the PARMS statement with an input
data set of possible start values, so the searach system can look
over more than one local optimum.

>2-) When I change distributions and values in "parms" I also get
>completely different parameter estimates.

Yes. Because when you change models, you change the
fundamental meaning of some of your constructs. So the
results change.

>3-) Worse than that, the predicted values are completely different from
>dependent variable values.

If your models do not give you meaningful results, then consider
scrapping the models and moving to something more functional.

Have you made your models too complex?

Do you really need ZI? models?

Do your data really warrant the use of such models?

>4-) Although, all these 5 models satisfy the convergence criterion, they
>ARE NOT consistent.
>
>What the use of doing Zero-Inflated Models, if they don't produce any
>results?

If your models do not match up with your data, then stick with
the data and scrap the models. There are certainly times when
ZIP andZINB models are very useful. But you may not have such
a case.

HTH,
David
--
David L. Cassell
mathematical statistician
Design Pathways
3115 NW Norwood Pl.
Corvallis OR 97330

__________________________________________________ _______________
All-in-one security and maintenance for your PC. Get a free 90-day trial!
http://clk.atdmt.com/MSN/go/msnnkwlo...id=msn_hotmail