Re: Reading records that are pointed to by other records

On Mon, 28 Feb 2005 10:54:07 -0800, Friar Broccoli <EliasRK@GMAIL.COM>
wrote:

>Hi World;
>
>I have some data which is in a proprietary format, most of
>which I understand at least in principle. The data has the
>following characteristics:
>
>1) ALL records are of fixed length (each one 92 bytes long).
> It is all in binary which is not a problem.
>
>2) Each record has a type identifier at the beginning of the
> record. There are about 5 types of record.
>
>3) All of the files I am reading can have all 5 types of
> records, and there is no referencing between separate
> files ... so each file can be handled in isolation from
> other files.
>
>4) One type of record (P0) is the Primary record type. A
> Primary record often contains pointers to one or more of
> the 4 other record types.
>
>5) ALL pointers simply point to the physical position of the
> record. Thus if a secondary record is at line/observation
> 88, then the pointer contains the value of 88.
>
>6) Secondary records (S1, S2, S3, S4) can themselves have
> pointers which always point to other secondary records of
> the same type.
>
>7) HERE IS MY PROBLEM. Most types of records can be located
> ANYWHERE in the original data file, so pointers can point
> either forward or back in the file. The record pointed
> to, can be arbitrarily far away.
>
>
>At present I have a kind of vague plan to import the data
>into 5 data sets adding some key fields which will contain,
>among other things, the original line position of the record
>in the source file. Thereafter try to stitch the data
>together, possibly with proc SQL.
>
>Since I only have a few months experience with SAS, and none
>of it recent, I am having difficulty formulating a plan for
>how to stitch the data together, after I have separated it
>into separate data sets. I am most concerned about how to
>read an observation that may be located anywhere in the
>secondary data set. I can't even figure out how to sort
>them to get them together.
>
>For example if a secondary record of type S2 (originally at
>line 188) points to another record of type S2 that was
>originally at line 74, how do I tell SAS to go get it?
>
>Suggestions (including complete different approaches) will
>be greatly appreciated.
>
>Cordially;
>
>Keith Elias




sounds like a good one for the dynamic memory management of the sas9
feature:
associative arrays
aka
hash tables

Provided there is sufficient memory, the tables would load into memory.
From there it should be reasonably fast to wander through the links.
More or less that sounds like how the system of the original "proprietary
format" would navigate. It definitely sounds like a nearly-recursive issue
that benefits from in-memory navigation.

??????
any comments (Paul, Jack, +++)

Peter Crawford