[kwlug-disc] wget and variable assignment

[Richard Weait]

On Thu, Jun 3, 2010 at 1:09 PM, Khalid Baheyeldin <kb at 2bits.com> wrote:
> On Thu, Jun 3, 2010 at 12:58 PM, Richard Weait <richard at weait.com> wrote:
>>
>> I have a simple screen-scrape to do.
>>
>> >From the command line it works fine
>>
>> wget -q -O - http://www.openstreetmap.org/stats/data_stats.html| grep
>> "<td>Number of users" | sed -e 's/[:a-zA-Z <>/:]//g'
>>
>> it returns the plain number
>>
>> 262086
>>
>> Cool, now to add it to a script
>>
>> This works fine
>> GETTEE=`wget -q -O -
>> http://www.openstreetmap.org/stats/data_stats.html| grep "<td>Number
>> of users" | sed -e 's/[:a-zA-Z <>/:]//g'`
>> echo "GETTEE = $GETTEE"
>>
>> gives:
>> GETTEE = 262086
>>
>> But.  I want to grab some other data from the same page, so I want to
>> wget once, then grep / sed a couple of times.  And I'm breaking it.
>> The page appears to have been stripped of its \n and so grepping the
>> line I want is failing.
>>
>> GETTEE=`wget -q -O - http://www.openstreetmap.org/stats/data_stats.html`
>> echo "GETTEE = $GETTEE"
>>
>> This returns a mess.
>>
>> The quick and dirty is to wget four times for four numbers, but I
>> don't want to do that.  How do I assign the wget to a variable and
>> keep \n ?
>
> This is not fair for whoever is hosting the server.

Right. That's why I'm here.

> Do the wget once, using
>
> wget -q -O /tmp/osm.html
>
> Then parse that file as many times as you want for whatever you want.

But no way to avoid all those disk calls?  The assignment to the
variable kills the \n or something?