Re: Basic question, I hope...SAS XML too

Sig,

Since SAS produces the following message:

WHERE 0 /* an obviously FALSE where clause */ ;

when the where clause is "0" and not when it is "1=2", the evidence would
suggest to me that SAS is smart enough to not bother to access the disk for
any of the records for the case when it knows a priori that the where
condition will be false. For the case when the where condition is 1=2, it
has to go to disk and load each record into the buffer, it then evaluates
the WHERE clause that common sense tells us will always be false and decides
to not bring record into the data set step PDV.

Regards,
-donh

>From: Sigurd Hermansen <HERMANS1@WESTAT.COM>
>Reply-To: Sigurd Hermansen <HERMANS1@WESTAT.COM>
>To: SAS-L@LISTSERV.UGA.EDU
>Subject: Re: Basic question, I hope...SAS XML too
>Date: Fri, 13 Jul 2007 14:23:34 -0400
>
>Au contraire. I understood your claim but didn't consider your evidence
>convincing.
>
>As you know, SAS may not release a table from a cache. A few seconds
>difference in reading a table a second time may be due to caching. After
>reversing the order of the two Data steps, however, I'm convinced of two
>facts: 1) SAS is reading the large dataset despite the a priori false
>condition; 2) you have really fast I/O or misleading times on your SAS
>log. My test showed too big a difference (around 6 minutes) in execution
>times to be the result of any other condition.
>S
>
>
>
>-----Original Message-----
>From: data _null_; [mailto:datanull@gmail.com]
>Sent: Friday, July 13, 2007 1:31 PM
>To: Sigurd Hermansen
>Cc: SAS-L@listserv.uga.edu
>Subject: Re: Basic question, I hope...SAS XML too
>
>
>You don't seem to understand or did not read the log I posted.
>
>WHERE 1=2 takes much longer to execute than
>WHERE 0
>
>therefore I conclude that SAS is processing "more something" for the
>first vs the second.
>
>Doesn't matter because the "correct" usage should be (OBS=0).
>
>On 7/13/07, Sigurd Hermansen <HERMANS1@westat.com> wrote:
> > I wouldn't jump to that conclusion based on what you have shown us...
> >
> > The SET ds1 ds2; construct works like a OUTER UNION CORR of two
> > datasets. In the example it writes NULL UNION sashelp.class to the
> > output dataset as expected. Remember that a dataset option operates on
>
> > one dataset. S
> >
> > -----Original Message-----
> > From: owner-sas-l@listserv.uga.edu
> > [mailtowner-sas-l@listserv.uga.edu]
> > On Behalf Of data _null_;
> > Sent: Friday, July 13, 2007 9:02 AM
> > To: d@dkvj.biz
> > Cc: SAS-L@listserv.uga.edu
> > Subject: Re: Basic question, I hope...SAS XML too
> >
> >
> > A simple test reveals that SAS is not able to determine that (1=2) is
> > obviously false.
> >
> > 36 options msglevel=i;
> > 37
> > 38 data work.class0;
> > 39 set sashelp.class;
> > 40 do _n_ = 1 to 1e6;
> > 41 output;
> > 42 end;
> > 43 run;
> >
> > NOTE: There were 19 observations read from the data set SASHELP.CLASS.
> > NOTE: The data set WORK.CLASS0 has 19000000 observations and 5
> > variables.
> > NOTE: DATA statement used (Total process time):
> > real time 9.53 seconds
> > cpu time 8.71 seconds
> >
> >
> > 44
> > 45 data work.class;
> > 46 set work.class0(where=(1=2)) sashelp.class;
> > 47 run;
> >
> > NOTE: There were 0 observations read from the data set WORK.CLASS0.
> > WHERE 1=2;
> > NOTE: There were 19 observations read from the data set SASHELP.CLASS.
> > NOTE: The data set WORK.CLASS has 19 observations and 5 variables.
> > NOTE: DATA statement used (Total process time):
> > real time 5.03 seconds
> > cpu time 4.59 seconds
> >
> >
> > 48
> > 49 data work.class;
> > 50 set work.class0(where=(0)) sashelp.class;
> > 51 run;
> >
> > NOTE: There were 0 observations read from the data set WORK.CLASS0.
> > WHERE 0 /* an obviously FALSE where clause */ ;
> > NOTE: There were 19 observations read from the data set SASHELP.CLASS.
> > NOTE: The data set WORK.CLASS has 19 observations and 5 variables.
> > NOTE: DATA statement used (Total process time):
> > real time 0.09 seconds
> > cpu time 0.00 seconds
> >
> >
> > 52
> > 53 data work.class;
> > 54 set work.class0(obs=0) sashelp.class;
> > 55 run;
> >
> > NOTE: There were 0 observations read from the data set WORK.CLASS0.
> > NOTE: There were 19 observations read from the data set SASHELP.CLASS.
> > NOTE: The data set WORK.CLASS has 19 observations and 5 variables.
> > NOTE: DATA statement used (Total process time):
> > real time 0.00 seconds
> > cpu time 0.00 seconds
> >
> >
> >
> >
> > On 7/12/07, David Johnson <d@dkvj.biz> wrote:
> > > This looks awfully like the process where I want to append data to
> > > an existing table but the table structures don't quite match. It
> > > doesn't
> >
> > > fix your root issue Alan, but it is a clean solution.
> > >
> > > Data TEST;
> > > Set SASHELP.CLASS( Obs = 0)
> > > TEST;
> > > Run;
> > >
> > >
> > > Semantically, the OBS option and the WHERE clause achieve the same
> > > result, but I need to ask if the WHERE clause will read all the data
>
> > > to verify it does not match, and the OBS option will simply take the
>
> > > header. On which basis, the OBS option may represent a more
> > > parsimonious solution.
> > >
> > > Kind regards
> > >
> > > David
> > >
> > >
> > > -----Original Message-----
> > > From: SAS(r) Discussion [mailto:SAS-L@LISTSERV.UGA.EDU]On Behalf Of
> > > Alan Churchill
> > > Sent: Thursday, 12 July 2007 3:19 PM
> > > To: SAS-L@LISTSERV.UGA.EDU
> > > Subject: Basic question, I hope...SAS XML too
> > >
> > >
> > > Ok, I have a routine that saves a SAS dataset from a .NET
> > > application.
> >
> > > However, SAS's XML engine does some weird things (such as reversing
> > > column order, not respecting schema, etc.). Therefore, I am creating
>
> > > a
> >
> > > routine to handle these thorny items by using the existing SAS
> > > dataset
> >
> > > as a template.
> > >
> > >
> > >
> > > My question: what is the best way to read an existing dataset into a
>
> > > new dataset, preserve its formats, labels, etc. and then read in the
>
> > > XML dataset and have it comply with the previous dataset's
> > > structure?
> > >
> > >
> > >
> > > As an example (which seems to work right now):
> > >
> > >
> > >
> > > libname outdata xml 'c:\temp\test2.xml' xmlmeta=schemadata
> > > xmltype=msaccess ;
> > >
> > >
> > >
> > > data outdata.test ;
> > >
> > > set sashelp.class;
> > >
> > > run;
> > >
> > >
> > >
> > > data test ;
> > >
> > > set sashelp.class (where=(1=2))
> > >
> > > outdata.test ;
> > >
> > > run;
> > >
> > >
> > >
> > > Is there a better way to accomplish this goal?
> > >
> > >
> > >
> > > Thanks,
> > >
> > > Alan
> > >
> > >
> > >
> > > Alan Churchill
> > > Savian "Bridging SAS and Microsoft Technologies"
> > > <http://www.savian.net/> www.savian.net
> > >
> >

__________________________________________________ _______________
http://liveearth.msn.com

Don Henderson wrote:
> Sig,
>
> Since SAS produces the following message:
>
> WHERE 0 /* an obviously FALSE where clause */ ;
>
> when the where clause is "0" and not when it is "1=2", the evidence would
> suggest to me that SAS is smart enough to not bother to access the disk for
> any of the records for the case when it knows a priori that the where
> condition will be false. For the case when the where condition is 1=2, it
> has to go to disk and load each record into the buffer, it then evaluates
> the WHERE clause that common sense tells us will always be false and decides
> to not bring record into the data set step PDV.
>
> Regards,
> -donh
>

Is SAS really loading the PDV and then evaluating 1=2? or is SAS
evaluating 1=2 and then deciding not to load the PDV (versus not even
evaluating 0)? I seem to remember long ago, when WHERE was new, that
it was faster than subsetting IF because SAS would read only enough
data to evaluate the condittion and then load the PDV only if it were
true.

-bwg