Ensuring Technical Readiness For Copilot in Microsoft 365
progress
1. Adding Progress Bars to Your Scripts http://oreilly.com/pub/h/943
Sign In/My Account | View Cart
Press
Book List Learning Lab PDFs O'Reilly GearNewsletters Jobs
Room
Buy the book!
HACK Adding Progress Bars to Your Scripts
Give a visual indication that a download is progressing smoothly
#18 The Code
[Discuss (3) | Link to this hack]
With all this downloading, it's often helpful to have some visual representation of its progress. In By Kevin Hemenway,
most of the scripts in this book, there's always a bit of visual information being displayed to the Tara Calishain
screen: that we're starting this URL here, processing this data there, and so on. These helpful bits October 2003
usually come before or after the actual data has been downloaded. But what if we want visual More Info
feedback while we're in the middle of a large MP3, movie, or database leech?
If you're using a fairly recent vintage of the LWP library, you'll be able to interject your own subroutine to run at regular
intervals during download. In this hack, we'll show you four different ways of adding various types of progress bars to
your current applications. To get the most from this hack, you should have ready a URL that's roughly 500 KB or larger;
it'll give you a good chance to see the progress bar in action.
The Code
The first progress bar is the simplest, providing only a visual heartbeat so that you can be sure things are progressing and
not just hanging. Save the following code to a file called progress_bar.pl and run it from the command line as perl
scriptnameURL, where URL is the online location of your appropriately large piece of sample data:
#!/usr/bin/perl -w
#
# Progress Bar: Dots - Simple example of an LWP progress bar.
# http://disobey.com/d/code/ or contact morbus@disobey.com.
#
# This code is free software; you can redistribute it and/or
# modify it under the same terms as Perl itself.
#
use strict; $|++;
my $VERSION = "1.0";
# make sure we have the modules we need, else die peacefully.
eval("use LWP 5.6.9;"); die "[err] LWP 5.6.9 or greater required.n" if $@;
# now, check for passed URLs for downloading.
die "[err] No URLs were passed for processing.n" unless @ARGV;
# our downloaded data.
my $final_data = undef;
# loop through each URL.
foreach my $url (@ARGV) {
print "Downloading URL at ", substr($url, 0, 40), "... ";
# create a new useragent and download the actual URL.
# all the data gets thrown into $final_data, which
# the callback subroutine appends to.
my $ua = LWP::UserAgent->new( );
my $response = $ua->get($url, ':content_cb' => &callback, );
print "n"; # after the final dot from downloading.
}
# per chunk.
sub callback {
my ($data, $response, $protocol) = @_;
$final_data .= $data;
print ".";
}
None of this code is particularly new, save the addition of our primitive progress bar. We use LWP's standard get method,
but add the :content_cb header with a value that is a reference to a subroutine that will be called at regular intervals as
our content is downloaded. These intervals can be suggested with an optional :read_size_hint, which is the number
of bytes you'd like received before they're passed to the callback.
1 di 5 20/06/2009 11.56
2. Adding Progress Bars to Your Scripts http://oreilly.com/pub/h/943
Search In this example, we've defined that the data should be sent to a subroutine named callback. You'll notice that the
Go routine receives the actual content, $data, that has been downloaded. Since we're overriding LWP's normal
$response->content or :content_file features, we now have to take full control of the data. In this hack, we
Hacks Site store all our results in $final_data, but we don't actually do anything with them.
• List of Titles
• Got a Hack? Most relevant, however, is the print statement within the callback routine. This is our first pathetic attempt at visual
• Suggestion Box feedback during the downloading process: every time a chunk of data gets sent our way, we spit out a dot. If the total data
size is sufficiently large, our screen will be filled with dots, dots, and more dots:
Resource Centers
Bioinformatics
Downloading URL at http://disobey.com/large_file.mov...
C/C++ ..........................................................................
Databases ..........................................................................
Digital Media ..........................................................................
Enterprise Development ..........................................................................
Game Development ..........................................................................
Java .....................................................................
Linux/Unix
Macintosh/OS X While useful, it's certainly not very pretty, and it can be especially disruptive for large files (the previous example is the
.NET output of downloading just 700 KB). Instead, how about we use a little primitive animation?
Open Source
Oracle If you've worked in the shell or installed various programs (or even a retail version of Linux), you may have seen rotating
Perl cursors built from ASCII letters. These cursors could start at , erase that character, draw a |, erase, /, erase, -, and then
Python to restart the loop. Individually, and without the benefit of a flipbook, these look pretty boring. Onscreen, however, they
Scripting
create a decent equivalent to an hourglass or spinning ball.
Security
Software Development
Modify the previous script, adding the highlighted lines:
SysAdmin/Networking
Web
...
Web Services
# our downloaded data.
Windows
my $final_data = undef;
Wireless
XML
# your animation and counter.
my $counter; my @animation = qw( | / - );
Book Series
Annoyances # loop through each URL.
CD Bookshelves foreach my $url (@ARGV)
Cookbooks ...
Developer's Notebooks
Hacks This initializes a counter and creates an array that contains the frames of our animations. As you can see, we use the same
Head First frames we discussed earlier. If you don't like `em, customize your own (perhaps . i l i). The last change we need to
In A Nutshell make is in our callback routine. Swap out the existing print "." with:
Missing Manuals
Pocket References print "$animation[$counter++]b";
Personal Trainer $counter = 0 if $counter == scalar(@animation);
Technology & Society
And that's it. For each chunk of data we receive, the next frame of the animation will play. When our counter is the same
Publishing Partners
as the number of frames, we reset and begin anew. Obviously, we can't show a readily apparent example of what this looks
Mandriva
No Starch Press
like, so try it at your leisure.
Paraglyph Press
PC Publishing
We can still do better, though. We've certainly removed the distracting dot distortion, but we're still left with only simple
Pragmatic Bookshelf output; we don't have raw information on how far we've gone and how far still to go. The following code provides a
SitePoint progress meter with a visual percentage bar, as well as a numerical reading:
Syngress Publishing
Online Publications
LinuxDevCenter.com
MacDevCenter.com
ONJava.com
ONLamp.com
OpenP2P.com
Perl.com
WebServices.XML.com
WindowsDevCenter.com
XML.com
Special Interest
Beta Chapters
Events
From the Editors List
Letters
MAKE
Open Books
tim.oreilly.com
Special Sales
Academic
Corporate Services
Government
Inside O'Reilly
2 di 5 20/06/2009 11.56
3. Adding Progress Bars to Your Scripts http://oreilly.com/pub/h/943
About O'Reilly
Bookstores #!/usr/bin/perl -w
Contact Us #
International # Progress Bar: Wget - Wget style progress bar with LWP.
Register Your Book # http://disobey.com/d/code/ or contact morbus@disobey.com.
User Groups # Original routine by tachyon at http://tachyon.perlmonk.org/
Writing for O'Reilly #
# This code is free software; you can redistribute it and/or
# modify it under the same terms as Perl itself.
#
use strict; $|++;
my $VERSION = "1.0";
# make sure we have the modules we need, else die peacefully.
eval("use LWP 5.6.9;"); die "[err] LWP 5.6.9 or greater required.n" if $@;
# now, check for passed URLs for downloading.
die "[err] No URLs were passed for processing.n" unless @ARGV;
# happy golucky variables.
my $final_data; # our downloaded data.
my $total_size; # total size of the URL.
# loop through each URL.
foreach my $url (@ARGV) {
print "Downloading URL at ", substr($url, 0, 40), "...n";
# create a new useragent and download the actual URL.
# all the data gets thrown into $final_data, which
# the callback subroutine appends to. before that,
# though, get the total size of the URL in question.
my $ua = LWP::UserAgent->new( );
my $result = $ua->head($url);
my $remote_headers = $result->headers;
$total_size = $remote_headers->content_length;
# now do the downloading.
my $response = $ua->get($url, ':content_cb' => &callback );
}
# per chunk.
sub callback {
my ($data, $response, $protocol) = @_;
$final_data .= $data;
print progress_bar( length($final_data), $total_size, 25, '=' );
}
# wget-style. routine by tachyon
# at http://tachyon.perlmonk.org/
sub progress_bar {
my ( $got, $total, $width, $char ) = @_;
$width ||= 25; $char ||= '=';
my $num_width = length $total;
sprintf "|%-${width}s| Got %${num_width}s bytes of %s (%.2f%%)r",
$char x (($width-1)*$got/$total). '>',
$got, $total, 100*$got/+$total;
}
You'll notice right off the bat that we've added another subroutine at the bottom of our code. Before we get into that, check
out our actual LWP request. Instead of just asking for the data, we first check the HTTP headers to see the size of the file
we'll be downloading. We store this size in a $total_size variable. It plays an important part in our new subroutine,
best demonstrated with a sample:
Downloading URL at http://disobey.com/large_file.mov...
|=============> | Got 422452 bytes of 689368 (61.28%)
This is sprintf magic at work, thanks to a little magic from tachyon over at Perl Monks
(http://www.perlmonks.org/index.pl?node_id=80749). As each chunk of data gets sent to our callback, the display is
updated both as a bar and as a byte count and percentage. It's a wonderful piece of work and my preferred progress bar as
of this writing. As you can see in the progress_bar line of the callback, you can modify the width as well as the
character.
So far, we've rolled our own, but there is a module on CPAN, Term::ProgressBar
(http://search.cpan.org/author/FLUFFY/Term-ProgressBar), that takes care of the lion's share of the work for us. It has a
bit more functionality than sprintf, such as titling the progress bar, including an ETA, and growing to the length of the
user's terminal width. Here it is in action:
3 di 5 20/06/2009 11.56
4. Adding Progress Bars to Your Scripts http://oreilly.com/pub/h/943
#!/usr/bin/perl -w
#
# Progress Bar: Term::ProgressBar - progress bar with LWP.
# http://disobey.com/d/code/ or contact morbus@disobey.com.
# Original routine by tachyon at http://tachyon.perlmonk.org/
#
# This code is free software; you can redistribute it and/or
# modify it under the same terms as Perl itself.
#
use strict; $|++;
my $VERSION = "1.0";
# make sure we have the modules we need, else die peacefully.
eval("use LWP 5.6.9;");
die "[err] LWP is not the required version.n" if $@;
eval("use Term::ProgressBar;"); # prevent word-wrapping.
die "[err] Term::ProgressBar not installed.n" if $@;
# now, check for passed URLs for downloading.
die "[err] No URLs were passed for processing.n" unless @ARGV;
# happy golucky variables.
my $final_data = 0; # our downloaded data.
my $total_size; # total size of the URL.
my $progress; # progress bar object.
my $next_update = 0; # reduce ProgressBar use.
# loop through each URL.
foreach my $url (@ARGV) {
print "Downloading URL at ", substr($url, 0, 40), "...n";
# create a new useragent and download the actual URL.
# all the data gets thrown into $final_data, which
# the callback subroutine appends to. before that,
# though, get the total size of the URL in question.
my $ua = LWP::UserAgent->new( );
my $result = $ua->head($url);
my $remote_headers = $result->headers;
$total_size = $remote_headers->content_length;
# initialize our progress bar.
$progress = Term::ProgressBar->new({count => $total_size, ETA => &return;
'linear'});
$progress->minor(0); # turns off the floating asterisks.
$progress->max_update_rate(1); # only relevant when ETA is used.
# now do the downloading.
my $response = $ua->get($url, ':content_cb' => &callback );
# top off the progress bar.
$progress->update($total_size);
}
# per chunk.
sub callback {
my ($data, $response, $protocol) = @_;
$final_data .= $data;
# reduce usage, as per example 3 in POD.
$next_update = $progress->update(length($final_data))if length($final_data) >= $ne
}
And here's its output:
Downloading URL at http://disobey.com/large_file.mov...
13% [======== ]9m57s Left
More examples are available in the Term::ProgressBar documentation.
Comment on this hack
You must be logged in to the O'Reilly Network to post a comment.
Showing
messages 1 through 3 of 3.
Warning
2005-11-06 04:02:02 tiberido [Reply | View]
4 di 5 20/06/2009 11.56
![Adding Progress Bars to Your Scripts http://oreilly.com/pub/h/943
Sign In/My Account | View Cart
Press
Book List Learning Lab PDFs O'Reilly GearNewsletters Jobs
Room
Buy the book!
HACK Adding Progress Bars to Your Scripts
Give a visual indication that a download is progressing smoothly
#18 The Code
[Discuss (3) | Link to this hack]
With all this downloading, it's often helpful to have some visual representation of its progress. In By Kevin Hemenway,
most of the scripts in this book, there's always a bit of visual information being displayed to the Tara Calishain
screen: that we're starting this URL here, processing this data there, and so on. These helpful bits October 2003
usually come before or after the actual data has been downloaded. But what if we want visual More Info
feedback while we're in the middle of a large MP3, movie, or database leech?
If you're using a fairly recent vintage of the LWP library, you'll be able to interject your own subroutine to run at regular
intervals during download. In this hack, we'll show you four different ways of adding various types of progress bars to
your current applications. To get the most from this hack, you should have ready a URL that's roughly 500 KB or larger;
it'll give you a good chance to see the progress bar in action.
The Code
The first progress bar is the simplest, providing only a visual heartbeat so that you can be sure things are progressing and
not just hanging. Save the following code to a file called progress_bar.pl and run it from the command line as perl
scriptnameURL, where URL is the online location of your appropriately large piece of sample data:
#!/usr/bin/perl -w
#
# Progress Bar: Dots - Simple example of an LWP progress bar.
# http://disobey.com/d/code/ or contact morbus@disobey.com.
#
# This code is free software; you can redistribute it and/or
# modify it under the same terms as Perl itself.
#
use strict; $|++;
my $VERSION = "1.0";
# make sure we have the modules we need, else die peacefully.
eval("use LWP 5.6.9;"); die "[err] LWP 5.6.9 or greater required.n" if $@;
# now, check for passed URLs for downloading.
die "[err] No URLs were passed for processing.n" unless @ARGV;
# our downloaded data.
my $final_data = undef;
# loop through each URL.
foreach my $url (@ARGV) {
print "Downloading URL at ", substr($url, 0, 40), "... ";
# create a new useragent and download the actual URL.
# all the data gets thrown into $final_data, which
# the callback subroutine appends to.
my $ua = LWP::UserAgent->new( );
my $response = $ua->get($url, ':content_cb' => &callback, );
print "n"; # after the final dot from downloading.
}
# per chunk.
sub callback {
my ($data, $response, $protocol) = @_;
$final_data .= $data;
print ".";
}
None of this code is particularly new, save the addition of our primitive progress bar. We use LWP's standard get method,
but add the :content_cb header with a value that is a reference to a subroutine that will be called at regular intervals as
our content is downloaded. These intervals can be suggested with an optional :read_size_hint, which is the number
of bytes you'd like received before they're passed to the callback.
1 di 5 20/06/2009 11.56](https://image.slidesharecdn.com/addingprogressbarstoyourscripts-090708093527-phpapp02/85/progress-1-320.jpg)
![Adding Progress Bars to Your Scripts http://oreilly.com/pub/h/943
Search In this example, we've defined that the data should be sent to a subroutine named callback. You'll notice that the
Go routine receives the actual content, $data, that has been downloaded. Since we're overriding LWP's normal
$response->content or :content_file features, we now have to take full control of the data. In this hack, we
Hacks Site store all our results in $final_data, but we don't actually do anything with them.
• List of Titles
• Got a Hack? Most relevant, however, is the print statement within the callback routine. This is our first pathetic attempt at visual
• Suggestion Box feedback during the downloading process: every time a chunk of data gets sent our way, we spit out a dot. If the total data
size is sufficiently large, our screen will be filled with dots, dots, and more dots:
Resource Centers
Bioinformatics
Downloading URL at http://disobey.com/large_file.mov...
C/C++ ..........................................................................
Databases ..........................................................................
Digital Media ..........................................................................
Enterprise Development ..........................................................................
Game Development ..........................................................................
Java .....................................................................
Linux/Unix
Macintosh/OS X While useful, it's certainly not very pretty, and it can be especially disruptive for large files (the previous example is the
.NET output of downloading just 700 KB). Instead, how about we use a little primitive animation?
Open Source
Oracle If you've worked in the shell or installed various programs (or even a retail version of Linux), you may have seen rotating
Perl cursors built from ASCII letters. These cursors could start at , erase that character, draw a |, erase, /, erase, -, and then
Python to restart the loop. Individually, and without the benefit of a flipbook, these look pretty boring. Onscreen, however, they
Scripting
create a decent equivalent to an hourglass or spinning ball.
Security
Software Development
Modify the previous script, adding the highlighted lines:
SysAdmin/Networking
Web
...
Web Services
# our downloaded data.
Windows
my $final_data = undef;
Wireless
XML
# your animation and counter.
my $counter; my @animation = qw( | / - );
Book Series
Annoyances # loop through each URL.
CD Bookshelves foreach my $url (@ARGV)
Cookbooks ...
Developer's Notebooks
Hacks This initializes a counter and creates an array that contains the frames of our animations. As you can see, we use the same
Head First frames we discussed earlier. If you don't like `em, customize your own (perhaps . i l i). The last change we need to
In A Nutshell make is in our callback routine. Swap out the existing print "." with:
Missing Manuals
Pocket References print "$animation[$counter++]b";
Personal Trainer $counter = 0 if $counter == scalar(@animation);
Technology & Society
And that's it. For each chunk of data we receive, the next frame of the animation will play. When our counter is the same
Publishing Partners
as the number of frames, we reset and begin anew. Obviously, we can't show a readily apparent example of what this looks
Mandriva
No Starch Press
like, so try it at your leisure.
Paraglyph Press
PC Publishing
We can still do better, though. We've certainly removed the distracting dot distortion, but we're still left with only simple
Pragmatic Bookshelf output; we don't have raw information on how far we've gone and how far still to go. The following code provides a
SitePoint progress meter with a visual percentage bar, as well as a numerical reading:
Syngress Publishing
Online Publications
LinuxDevCenter.com
MacDevCenter.com
ONJava.com
ONLamp.com
OpenP2P.com
Perl.com
WebServices.XML.com
WindowsDevCenter.com
XML.com
Special Interest
Beta Chapters
Events
From the Editors List
Letters
MAKE
Open Books
tim.oreilly.com
Special Sales
Academic
Corporate Services
Government
Inside O'Reilly
2 di 5 20/06/2009 11.56](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)