[kwlug-disc] wget and variable assignment

[Khalid Baheyeldin]

On Thu, Jun 3, 2010 at 12:58 PM, Richard Weait <richard at weait.com> wrote:

> I have a simple screen-scrape to do.
>
> >From the command line it works fine
>
> wget -q -O - http://www.openstreetmap.org/stats/data_stats.html|<http://www.openstreetmap.org/stats/data_stats.html%7C>grep
> "<td>Number of users" | sed -e 's/[:a-zA-Z <>/:]//g'
>
> it returns the plain number
>
> 262086
>
> Cool, now to add it to a script
>
> This works fine
> GETTEE=`wget -q -O -
> http://www.openstreetmap.org/stats/data_stats.html|<http://www.openstreetmap.org/stats/data_stats.html%7C>grep "<td>Number
> of users" | sed -e 's/[:a-zA-Z <>/:]//g'`
> echo "GETTEE = $GETTEE"
>
> gives:
> GETTEE = 262086
>
> But.  I want to grab some other data from the same page, so I want to
> wget once, then grep / sed a couple of times.  And I'm breaking it.
> The page appears to have been stripped of its \n and so grepping the
> line I want is failing.
>
> GETTEE=`wget -q -O - http://www.openstreetmap.org/stats/data_stats.html`<http://www.openstreetmap.org/stats/data_stats.html%60>
> echo "GETTEE = $GETTEE"
>
> This returns a mess.
>
> The quick and dirty is to wget four times for four numbers, but I
> don't want to do that.  How do I assign the wget to a variable and
> keep \n ?
>

This is not fair for whoever is hosting the server.

Do the wget once, using

wget -q -O /tmp/osm.html

Then parse that file as many times as you want for whatever you want.
-- 
Khalid M. Baheyeldin
2bits.com, Inc.
http://2bits.com
Drupal optimization, development, customization and consulting.
Simplicity is prerequisite for reliability. --  Edsger W.Dijkstra
Simplicity is the ultimate sophistication. --   Leonardo da Vinci
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://kwlug.org/pipermail/kwlug-disc_kwlug.org/attachments/20100603/57fb5a71/attachment.htm>