how to get the values saved in variables?

Hi, I work like a robot today. My job is to visit a webpage, copy
several numbers, paste them to a text file. Then another webpage......
There are about 100 URLs. I decided to download all webpages and
process those files to extract the numbers.

I used a program called url2file to download webpages. However, the
numbers I need to extract are not there. I got something like

<script>document.write(v1)</script>

where v1 holds the number.

Is it possible to get the values in javascript variables without manual
work?

Thank you.

Ben

Ben wrote:
> Hi, I work like a robot today. My job is to visit a webpage, copy
> several numbers, paste them to a text file. Then another webpage......
> There are about 100 URLs. I decided to download all webpages and
> process those files to extract the numbers.
>
> I used a program called url2file to download webpages. However, the
> numbers I need to extract are not there. I got something like
>
> <script>document.write(v1)</script>
>
> where v1 holds the number.
>
> Is it possible to get the values in javascript variables without manual
> work?
>
> Thank you.
>
> Ben

I doubt you'll be able to get the numbers from javascript unless
they're stored in a variable, or maybe by redeclaring document.write().
Anyhow, in order to complete the task you mention automatically, I'd
recommend you to create a PHP/bash/perl/ruby/... script to parse each
file with a regular expression and get the value between ( ).

In that case, I guess you'll get more help in another list.

Best,
- Guillermo.

Ben wrote:

> I used a program called url2file to download webpages. However, the
> numbers I need to extract are not there. I got something like
>
> <script>document.write(v1)</script>
>
> where v1 holds the number.
>
> Is it possible to get the values in javascript variables without manual
> work?

IE on Windows can be automated with script so you could write a script
to fire up IE, load a URL, read out a value, load the next URL.

Another way might be to use HTTPUnit <http://www.httpunit.org/>.


--

Martin Honnen
http://JavaScript.FAQTs.com/

I am working on a project to collect data from websites. Server side
scripting is impossible.

There's no problem if the data were returned in plain html files. For
javascript variables, Is there any way to simulate the web browser to
interpret the code in downloaded files so that we can add some code to
write the values out to a text file?

Guillermo Rauch wrote:
> Ben wrote:
> > Hi, I work like a robot today. My job is to visit a webpage, copy
> > several numbers, paste them to a text file. Then another webpage......
> > There are about 100 URLs. I decided to download all webpages and
> > process those files to extract the numbers.
> >
> > I used a program called url2file to download webpages. However, the
> > numbers I need to extract are not there. I got something like
> >
> > <script>document.write(v1)</script>
> >
> > where v1 holds the number.
> >
> > Is it possible to get the values in javascript variables without manual
> > work?
> >
> > Thank you.
> >
> > Ben
>
> I doubt you'll be able to get the numbers from javascript unless
> they're stored in a variable, or maybe by redeclaring document.write().
> Anyhow, in order to complete the task you mention automatically, I'd
> recommend you to create a PHP/bash/perl/ruby/... script to parse each
> file with a regular expression and get the value between ( ).
>
> In that case, I guess you'll get more help in another list.
>
> Best,
> - Guillermo.

> IE on Windows can be automated with script so you could write a script
> to fire up IE, load a URL, read out a value, load the next URL.

This is the way I am looking for. I searched IE automation and got tons
of links. Don't know which one is close to what I am looking for. Start
with MSDN forums...

Martin Honnen wrote:
> Ben wrote:
>
> > I used a program called url2file to download webpages. However, the
> > numbers I need to extract are not there. I got something like
> >
> > <script>document.write(v1)</script>
> >
> > where v1 holds the number.
> >
> > Is it possible to get the values in javascript variables without manual
> > work?
>
> IE on Windows can be automated with script so you could write a script
> to fire up IE, load a URL, read out a value, load the next URL.
>
> Another way might be to use HTTPUnit <http://www.httpunit.org/>.
>
>
> --
>
> Martin Honnen
> http://JavaScript.FAQTs.com/

have you tried wget ?? it is very easy to use, and it can download
websites from list to one file.

On Jan 2, 5:29 pm, "Ben" <laser...@gmail.com> wrote:
> Hi, I work like a robot today. My job is to visit a webpage, copy
> several numbers, paste them to a text file. Then another webpage......
> There are about 100 URLs. I decided to download all webpages and
> process those files to extract the numbers.
>
> I used a program called url2file to download webpages. However, the
> numbers I need to extract are not there. I got something like
>
> <script>document.write(v1)</script>
>
> where v1 holds the number.
>
> Is it possible to get the values in javascript variables without manual
> work?
>
> Thank you.
>
> Ben

On 2 Jan 2007, laserbin@gmail.com wrote:

> Hi, I work like a robot today. My job is to visit a webpage, copy
> several numbers, paste them to a text file. Then another webpage......
> There are about 100 URLs. I decided to download all webpages and
> process those files to extract the numbers.
>
> I used a program called url2file to download webpages. However, the
> numbers I need to extract are not there. I got something like
>
> <script>document.write(v1)</script>
>
> where v1 holds the number.

This should work on most OSs:

curl URL | perl -ne'm/document\.write\((.*)\)/ && print "$1\n"'

It prints everything between the parenthesis of document.write(...)
and nothing else, separating the values with a newline.

If you are on Windows, you may want to try cygwin, which will let you
do the command above easily (as long as you've installed Perl and curl).

Ted