help for reading a binary file

Hi,everybody here!

I'm using a Fortran code based on the 386|DOS-Extender, a 32-bit
protected mode development tool developped by Phar Lap Software, Inc.

It will generate a binary data file after running. Now I need to read
out the data. The manual mentioned the format of this as follows:
================================================== ==============
The file has two records. In the 1st record, there are three variables
(the first one is a 4-byte integer and the other two are 8-byte double
precision real variables). The second record contains 6*1000 8-byte
double precision real variables (array(I,J),I=1,6),J=1,1000).
================================================== ==============

I edited a code using lf95 as follows:
================================================== ==============
program test

double precision bb,cc,array(6000)

integer*4 aa

character*32 testinputfile

testinputfile = 1.dat'

open(22,file=testinputfile ,status='old',form='unformatted')

read(22,*)aa,bb,cc,(array(i),i=1,6000)

write(5,*)aa,bb,cc,(array(i),i=1,6000)

end
================================================== ==============

Running my code, I got a file, fort.5. Though the total number of the
gotten data is right as my expection, but the values look very strange.
For example, the gotten aa is 858980352 which should be 1000. Other
values like 3.498815625006737E-310 or -2.280122331273626E+16, their
orders are very large.

Could anyone helps me out? Many thanks in advance!

phywho schreef:

> Hi,everybody here!
>
> Running my code, I got a file, fort.5. Though the total number of the
> gotten data is right as my expection, but the values look very strange.
> For example, the gotten aa is 858980352 which should be 1000. Other
> values like 3.498815625006737E-310 or -2.280122331273626E+16, their
> orders are very large.
>
> Could anyone helps me out? Many thanks in advance!

The problem is that _binary_ files and _unformatted_ files are not the
same thing.
If the file you read is indeed binary, then there are no extra bytes
that identify
the records. If it is an unformatted file, then there are.

One way to try and read this file is by using form="binary" or
form="transparant"
(the value is not portable). There are other solutions too (see
http://flibs.sf.net, for
instance for a simple implementation of stream I/O - the kind of I/O
you need).

(No time for more details right now - I am sure others can help you
further)
Regards,

Arjen

In message <1150812683.140828.43380@c74g2000cwc.googlegroups. com>, Arjen
Markus <arjen.markus@wldelft.nl> writes
>One way to try and read this file is by using form="binary" or
>form="transparant"
>(the value is not portable). There are other solutions too (see
>http://flibs.sf.net, for
>instance for a simple implementation of stream I/O - the kind of I/O
>you need).

And if you are willing to use g95, you will find F2003-Standard stream
I/O already implemented. This will enable you to read a binary stream
byte-by-byte. Several other compilers implement something similar, but
not standard-conforming.

--
Clive Page

"phywho" <phywho@gmail.com> wrote in message
news:1150811893.096266.323250@i40g2000cwc.googlegr oups.com...
> Hi,everybody here!
>
> I'm using a Fortran code based on the 386|DOS-Extender, a 32-bit
> protected mode development tool developped by Phar Lap Software, Inc.
>
> It will generate a binary data file after running. Now I need to read
> out the data. The manual mentioned the format of this as follows:
> ================================================== ==============
> The file has two records. In the 1st record, there are three variables
> (the first one is a 4-byte integer and the other two are 8-byte double
> precision real variables). The second record contains 6*1000 8-byte
> double precision real variables (array(I,J),I=1,6),J=1,1000).
> ================================================== ==============
>
> I edited a code using lf95 as follows:
> ================================================== ==============
> program test
> double precision bb,cc,array(6000)
> integer*4 aa
> character*32 testinputfile
> testinputfile = 1.dat'

Missing apostrophe.

> open(22,file=testinputfile ,status='old',form='unformatted')
> read(22,*)aa,bb,cc,(array(i),i=1,6000)
> write(5,*)aa,bb,cc,(array(i),i=1,6000)
> end
> ================================================== ==============
>
> Running my code, I got a file, fort.5. Though the total number of the
> gotten data is right as my expection, but the values look very strange.
> For example, the gotten aa is 858980352 which should be 1000. Other
> values like 3.498815625006737E-310 or -2.280122331273626E+16, their
> orders are very large.
>
> Could anyone helps me out? Many thanks in advance!

Won't you need two READs?
One for aa, bb, cc, and the other for array.
The above READ will read only one record.

robin wrote:

>
> Won't you need two READs?
> One for aa, bb, cc, and the other for array.
> The above READ will read only one record.


Thank you all, Arjen, Clive Page and Robin, very much!

To Arjen: The total size of the data file is 10 bytes more for that of
the required data. I guess that there are some extra bytes to identify
the records. I will try to do some tests according to your advice, but
I don't know if I could solve it so soon.

To Clive Page: in fact, I'm a new hand in Fortran, so I just know some
basic operations of lf95, which is on hand. I will try to use the g95.

To Robin:In the case of using form='unformatted', I could read out data
when I use only one READ, but failed when two READs were used. In the
case of using form='binary', it doesn't care one or two READs are used
and I can read out the data, which are different from the 'unformatted'
ones. I don't know why.

As I mentioned, I just touched Fortran and I have such an urgent task.
Any detailed help such as some lines of codes will be appreciated a lot!

phywho wrote:

(snip)

> To Robin:In the case of using form='unformatted', I could read out data
> when I use only one READ, but failed when two READs were used. In the
> case of using form='binary', it doesn't care one or two READs are used
> and I can read out the data, which are different from the 'unformatted'
> ones. I don't know why.

UNFORMATTED has been part of Fortran for a long time, and is record
oriented. You can read less than a whole record, and the next
READ will still start on the next record.

form='binary' is non-standard, at least as of Fortran 95, but is
sometimes needed to read non-Fortran files.

If you are only reading back files that you write from Fortran programs
form='unformatted' should be fine, and also standard.

-- glen

phywho wrote:
> Hi,everybody here!
>
> I'm using a Fortran code based on the 386|DOS-Extender, a 32-bit
> protected mode development tool developped by Phar Lap Software, Inc.
>
> It will generate a binary data file after running. Now I need to read
> out the data. The manual mentioned the format of this as follows:
> ================================================== ==============
> The file has two records. In the 1st record, there are three variables
> (the first one is a 4-byte integer and the other two are 8-byte double
> precision real variables). The second record contains 6*1000 8-byte
> double precision real variables (array(I,J),I=1,6),J=1,1000).
> ================================================== ==============
>
> I edited a code using lf95 as follows:
> ================================================== ==============
> program test
>
> double precision bb,cc,array(6000)
>
> integer*4 aa
>
> character*32 testinputfile
>
> testinputfile = 1.dat'
>
> open(22,file=testinputfile ,status='old',form='unformatted')
>
> read(22,*)aa,bb,cc,(array(i),i=1,6000)

If your file is unformatted, try

read(22)aa,bb,cc,(array(i),i=1,6000)

>
> write(5,*)aa,bb,cc,(array(i),i=1,6000)
>
> end
> ================================================== ==============
>
> Running my code, I got a file, fort.5. Though the total number of the
> gotten data is right as my expection, but the values look very strange.
> For example, the gotten aa is 858980352 which should be 1000. Other
> values like 3.498815625006737E-310 or -2.280122331273626E+16, their
> orders are very large.
>
> Could anyone helps me out? Many thanks in advance!

In message <1150877716.209523.233520@y41g2000cwy.googlegroups .com>,
phywho <phywho@gmail.com> writes
>To Clive Page: in fact, I'm a new hand in Fortran, so I just know some
>basic operations of lf95, which is on hand. I will try to use the g95.

You can get g95 from www.g95.org and it's pretty easy to get going, but
the documentation is still basic.

>
>As I mentioned, I just touched Fortran and I have such an urgent task.
>Any detailed help such as some lines of codes will be appreciated a lot!

Some notes I wrote here http://www.star.le.ac.uk/%7Ecgp/streamIO.html
give an example of reading a .DBF file. Hope that helps.

--
Clive Page

[reading back an unformatted data file with a program compiled with a
different Fortran does not work]

> To Arjen: The total size of the data file is 10 bytes more for that of
> the required data. I guess that there are some extra bytes to identify
> the records. I will try to do some tests according to your advice, but
> I don't know if I could solve it so soon.

You can not expect to read unformatted files written by compiler X with
compiler Y.

> To Clive Page: in fact, I'm a new hand in Fortran, so I just know some
> basic operations of lf95, which is on hand. I will try to use the g95.

Look at the data file with a file editor. Locate the non data bytes
before and after each record. Use stream i/o as in g95. Read the
non-data prefix bytes. Read the data. Read the non-data suffix bytes.
Lather, rinse, repeat.

> As I mentioned, I just touched Fortran and I have such an urgent task.
> Any detailed help such as some lines of codes will be appreciated a lot!

"Adding manpower to a late software project makes it later." (Fred
Brooks).

"phywho" <phywho@gmail.com> wrote in message
news:1150877716.209523.233520@y41g2000cwy.googlegr oups.com...
>
> robin wrote:
>
> > Won't you need two READs?
> > One for aa, bb, cc, and the other for array.
> > The above READ will read only one record.
> Thank you all, Arjen, Clive Page and Robin, very much!
>
> To Arjen: The total size of the data file is 10 bytes more for that of
> the required data. I guess that there are some extra bytes to identify
> the records. I will try to do some tests according to your advice, but
> I don't know if I could solve it so soon.
>
> To Clive Page: in fact, I'm a new hand in Fortran, so I just know some
> basic operations of lf95, which is on hand. I will try to use the g95.
>
> To Robin:In the case of using form='unformatted', I could read out data
> when I use only one READ,

You said that the file had two records. Therefore, two READs
are required.

phywho wrote:
> robin wrote:
>
> >
> > Won't you need two READs?
> > One for aa, bb, cc, and the other for array.
> > The above READ will read only one record.
>
>
> Thank you all, Arjen, Clive Page and Robin, very much!
>
> To Arjen: The total size of the data file is 10 bytes more for that of
> the required data. I guess that there are some extra bytes to identify
> the records. I will try to do some tests according to your advice, but
> I don't know if I could solve it so soon.

10 extra bytes seems strange. I can see a 4 byte prefix for each record
or a 2 byte prefix and a 2 byte suffix for each record. IIRC, old Lahey
Fortran also had (optional) file header bytes.

HTH

Many thanks to all, especially to Chandler!

With the clew from you, I have read the data which seems reasonable by
applying a 2 byte prefix and a 3 byte suffix for a record. It's a
little strange, but it works. I need to check the data carefully again
and find out what are the contents and variable types of the prefixes
and suffixes. As Einstein said, the more we learn, the more we don't
know.

Fortunately, in mordern time we can have a great community online like
here, though we don't know each other. It's amazing...


e p chandler wrote:

>
> 10 extra bytes seems strange. I can see a 4 byte prefix for each record
> or a 2 byte prefix and a 2 byte suffix for each record. IIRC, old Lahey
> Fortran also had (optional) file header bytes.
>
> HTH

You should use FORM='BINARY', ACCESS='SEQUENTIAL'.
You could even use ACCESS='DIRECT', FORM=UNFORMATTED', and specify the
recored length (as either 1 or 4-byte multiples of bytes, depending on
the compiler default for LRECL interpretation).

You also need a binary file dump program as a tool to find out what
files formatting exists in unknown data. I suggest FM.com, (DOS, or DOS
in Windows) or our own FMNT.exe (free windows version).

Terence Wright

> I'm using a Fortran code based on the 386|DOS-Extender, a 32-bit
> protected mode development tool developped by Phar Lap Software, Inc.
>
> It will generate a binary data file after running. Now I need to read
> out the data. The manual mentioned the format of this as follows:
> ================================================== ==============
> The file has two records. In the 1st record, there are three variables
> (the first one is a 4-byte integer and the other two are 8-byte double
> precision real variables). The second record contains 6*1000 8-byte
> double precision real variables (array(I,J),I=1,6),J=1,1000).
> ================================================== ==============
>
> Could anyone helps me out? Many thanks in advance!

<tbwright@cantv.net> wrote:

> You should use FORM='BINARY', ACCESS='SEQUENTIAL'.

Please be aware that this is an extension not supported by all
compilers. Among those that do support it, the exact spelling varies.

The functionality is finally standardized (as access='stream') in f2003.
That standardization is overdue in my opinion. It is painful to attempt
to deal with this kind of thing with any reasonable portability prior to
f2003.

--
Richard Maine | Good judgement comes from experience;
email: last name at domain . net | experience comes from bad judgement.
domain: summertriangle | -- Mark Twain

Robin noted briefly his opinion that if there are two "records" (which
I interpret to mean two strings each ending in an cr-lf terminator),
then two reads are required.

This is usual, but not strictly correct. You can read external file
data as a character stream, with unformatted sequential reads, or in
blocks using unformatted direct access, where one single read can
capture the entire file.
Reading as a stream of characters, and searching for the known
terminators is a useful common way to process someone else's data when
the file format in unclear.

Terence Wright
..