Re: Decision Tree refuses to grow

Dropping the significance value may have the opposite of the effect that
you expect. Generally it takes a 'purer' separation of true events from
others to attain 1% as opposed to a 5% Type 1 error 'significance'. A
small proportion of true events in data makes it even harder (due to
'long-tail' distributions of errors).

SAS/EM takes statistical significance seriously and won't produce
results in some situations unless the user explicitly increases the
acceptable level of Type 1 error. I would prefer a 'decision cost' basis
for exploratory data analyses that do not pretend to conduct a
hypothesis test, but I understand why SAS implements decision trees this
way.
Sig

-----Original Message-----
From: owner-sas-l@listserv.uga.edu [mailtowner-sas-l@listserv.uga.edu]
On Behalf Of amw5gster@gmail.com
Sent: Wednesday, November 29, 2006 11:50 AM
To: sas-l@uga.edu
Subject: Decision Tree refuses to grow


Howdy,

Silly question that's likely to show I'm overlooking something simple,
but I'm stumped. I have a dset of approx 8M observations and I'm trying
to grow an EMiner decision tree on a binary target variable. There are
about 20 independent variables, mostly interval (dates), but some
nominal, a few binary and one ordinal. The proportion of true events is
about 12%. I have not set any prior probabilities, nor profit/cost
values.

The tree runs, but returns no splits. It just won't grow. I've tried
dropping the signif value to .00001, using upwards of 11 maximum
branches and my max depth to 10. I also tried having the tree build on
as few as 2 IVs.

I was able to build a tree when I took a sample of 100K records and
forced the %age of true events in the sample to be 50%. Naturally I
don't want to misrepresent the proportion, and I figured that 12% wasn't
terribly rare for a d-tree.

Am I outright doing something wrong or is this expected behavior?