Re: transferring a large pc sas dataset to unix

Dear SAS-L-ers,

Jennifer R. Holdcraft posted the following interesting problem:

> I was wondering if someone could help me with the transfer of
> a large (around 50 gig) sas dataset on a PC
> to UNIX? The original file was on UNIX (AIX) and due to some space
> concerns on the server at the time,
> someone copied the file to a PC to make some modifications.
> We have a Samba connection - so the first time around, they
> just made a libname connection to the unix server and used a
> set statement to bring
> in the dataset. This took HOURS! So now that the
> modifications are
> done, we were trying to find a
> quicker way to bring the dataset back to UNIX than another
> data step. Someone tried just FTP'ing the file over and we
> opened it up and everything looked fine, but when we tried to
> append the next days results to that file we got an error
> message complaining that it was made with another operating system.
> Oops! We called SAS and they suggested the proc copy -
> transport file
> - FTP - proc copy route, but
> so far we haven't been able to get the syntax right (even
> though we've been trying to mimic an example in the SAS
> documentation) - it is complaining about names being too
> long, and we when we are making the libname statement for the
> xport engine we don't know what (if?) we are supposed to put
> as the filename extension, etc..
>
> Any help you can give would be greatly appreciated!
>
> Jennifer R. Holdcraft
>
Jennifer, WAIT! STOP! Before we go any further, please, please, please
tell me that they did not delete the data set that was FTP-ed to the
UNIX server! Check on it; I'll wait here...
....
....
....
Back already? It's still there? Good; then your task is half done; the
rest will be easy! Here is why:

You wrote:

> data step. Someone tried just FTP'ing the file over and we
> opened it up and everything looked fine, but when we tried to
> append the next days results to that file we got an error
> message complaining that it was made with another operating system.
> Oops!
>

So, you have a living, breathing, SAS data set in PC file format living
on a UNIX server. SAS uses Cross Environment Data Access (CEDA) engines
to read the data set--that is why you were able to write: "...we opened
it up and everything looked fine...". CEDA engines allow you to read a
SAS data set in a foreign data representation that was FTP-ed between
(most) operating systems. However, there are limitations, such as you
not being able to update the foreign SAS data set on the new computer
platform. All of this is discussed in loving detail in Bruce Gilsen's
brilliant SUGI 31 paper:

CEDA - The Invisible Hand
http://www2.sas.com/proceedings/sugi31/225-31.pdf

Okay, so, getting back to your problem: Now that you have that big
clunker on UNIX, you can simply execute SAS on UNIX to copy the SAS data
set elsewhere. The COPY Procedure, running on UNIX will translate the
foreign data representation to UNIX data representation automatically.
Something as simple as this should work:

libname ftpdir "/home/someone/ftpdir";
libname jendir "/home/jennifers/sasdir";

proc copy in=ftpdir out=jendir;
select clunker;
run;

Executing the SAS code, above, clunker.sas7bdat in Windows format in the
"/home/someone/ftpdir" directory will be copied to the
"/home/jennifers/sasdir" directory. When all is said and done (by SAS),
clunker.sas7bdat in the "/home/jennifers/sasdir" directory will be in
proper UNIX format, ready, willing, and able to have new observations
appended to it.

BTW, you can determine whether data sets are in a non-native format by
looking at the header information in the CONTENTS Procedure. But, you
will learn all about this and more in Bruce's paper.

Jennifer, best of luck to you in getting humongous SAS data sets from
the narrow confines of Windows to the wide open spaces of UNIX!


I hope that this suggestion proves helpful now, and in the future!

Of course, all of these opinions and insights are my own, and do not
reflect those of my organization or my associates. All SAS code and/or
methodologies specified in this posting are for illustrative purposes
only and no warranty is stated or implied as to their accuracy or
applicability. People deciding to use information in this posting do so
at their own risk.

++++++++++++++++++++++++++++++++++++++++++++++++++ +++++++++
Michael A. Raithel
"The man who wrote the book on performance"
E-mail: MichaelRaithel@westat.com

Author: Tuning SAS Applications in the MVS Environment

Author: Tuning SAS Applications in the OS/390 and z/OS Environments,
Second Edition
http://www.sas.com/apps/pubscat/book...tid=1&pc=58172

Author: The Complete Guide to SAS Indexes
http://www.sas.com/apps/pubscat/book...tid=1&pc=60409

++++++++++++++++++++++++++++++++++++++++++++++++++ +++++++++
You can't have everything. Where would you put it? - Steven Wright
++++++++++++++++++++++++++++++++++++++++++++++++++ +++++++++