Kludgesoft

Webpack

What is it?
News
Features
Download & Usage
Requirements
History
To-do
Contact/Feedback

What is it?

Webpack is an open-source command-line tool for automatically packing websites by shrinking them down without affecting the way they look or behave.Webpack is also useful for losslessly shrinking image collections and locating corrupted files. Webpack works by stripping unnecesary information from & optimising the compression of images, and removing comments/whitespace from  html. This makes for a faster website for your users, and lower bandwidth usage/costs for you! (oh, and it's free - both as in speech and as in beer).

News

4-10-2004 - Webpack 0.6b released
  • WAY more sanity checking
    • check if source or destination is the root directory
    • check if output is same as, subdirectory of or parent of current directory
    • now checks all stupid combinations of source/destination directories I could imagine
  • enable force overwrite (-f) option
  • works with (more) paths with spaces in them
  • add support for HTML::Clean for html compression
  • add support for advancecomp (compression optimizer):
    • use advdef to recompress *.gz (and *.png if pngcrush not installed)
    • use advzip to recompress *.zip
    • brute mode (-b) enables -4 (compress extreme) option on advdef/advzip
13-9-2004 - Webpack 0.5b released
  • catch (more) pngcrush failures
  • minor cleanups
  • more sanity checking
  • implement process-existing-files-only-if-source-newer-than-target (-u) - now can use in an automated way to keep a website updated
  • implement force overwrite (-f) (hidden for the moment)
25-07-2004 - Freshmeat Announcement
Webpack got announced on Freshmeat and there's been lots of hits but not many downloads, so to make it easier there is now linux-x86 and cygwin "packages" which include both webpack itself and enough of the pre-compiled requirements to get you going. Oh and yes, future versions will have a bit more documentation :-)

Features

  • Losslessly packs GIF, JPG & PNG images (images are pixel-for-pixel identical, no quality loss)
  • Strips HTML of unneeded comments/whitespace (with htmlcrunch)
  • Always copies the original file if it can't be improved (files never get bigger)
  • Prints lots of useful statistics, including size breakdown by image type
  • Maintains original time/date stamps of copied files
  • quiet mode -q (show only errors)
  • brute mode -b (try really hard - makes some files even smaller)
  • update mode -u (update existing destination files if the source file is newer)

Webpack is in development, but works for us, and we need testers (that'd be you!). Webpack is "safe" in that it is designed to create a compressed replica of a directory tree, so your original files are not touched. Any files it doesn't know what to do with are copied verbatim. Basically you run webpack on your sites original files and upload the webpack-ed version to your webserver - easy!

Download & Usage Instructions

Download webpack 0.6b (webpack shell script & README)

Webpack is released under the terms of the GNU General Public License (GPL).

Packages:
These contain webpack, jpegtran, gifsicle & htmlcrunch executables (for if you don't want to download & compile them separately)
Download webpack 0.6b linux-x86 package (from and/or compiled on RH9)
 Download webpack 0.6b cygwin package (get this if you use cygwin on windows)

Basic usage

To install, simply copy "webpack" into ~/bin or /usr/local/bin (whichever is your preference), and make sure you have at least some of the requirements below.
To use:
a) change into the directory you want to 'webpack' eg "cd ~/mysite"
b) do a "webpack <output directory>" eg "webpack ~/mysitepacked"
.. and that's it - webpack will create a packed replica of the current directory (and subdirectories) in <output directory>.
Use the -b (brute) option if you want absolute maximum file reduction (makes JPG's a bit slower to process and PNG's a lot slower, no affect on other types yet).
Use the -q (quiet) option if you want to see only warnings/errors.
Webpack will remind you what to do with no arguments, and there is a bit of error checking in there to stop accidents. (at the moment it won't overwrite existing files so you can't do much harm).
Feedback/suggestions/bugreports are encouraged!

Requirements

Webpack relies on several readily available command line tools to process various types of files. You will need to install some (preferably all) of these for it to be useful. Most of them either come with your linux distribution or are easy to compile (usually ./configure; make; make install or less).
At an absolute minimum, install jpegtran and giftrans (which come with most linux distributions anyway), but if you have a lot of gifs to process, we strongly recommend gifsicle for processing them, as it significantly outperforms what giftrans can do (which isn't much).
If you don't have some of these tools installed webpack will just copy the particular type of file that tool would have processed.

History

Webpack 0.1 alpha 18/04/2004
  • first version (unreleased)
  • processes gif, jpg & png files
  • imited sanity checking, but won't overwrite anything
Webpack 0.2 alpha 8/05/2004
  • first public release
  • track input & output filesizes
    • print #files, inbytes, outbytes & % reduction
    • separate stats for images/html/remainder/total
    • break down to individual image/file type
    • print % and byte reduction for each file
  • check if file got bigger (or was same size) and copy original if it did (prints "!")
    • for jpg, also try non-progressive to see if it's smaller before copying original (prints "*")
  • try to maintain flags, modified time etc if original was copied
Webpack 0.3 beta 5/07/2004
  • it's now beta because it works for me :)
  • implemented options/flags logic
  • use HTMLCrunch for compressing html (results not that good unfortunately)
  • implement brute mode (-b)
    • always try both interlaced & non-interlaced for jpgs
    • supply -brute flag to pngcrush
  • recompress bzip2 files if -9 wasn't used
    • checks file for what method was used (not sure if this is 100% reliable?)
Webpack 0.4 beta 19/07/2004
  • now also work with gifsicle for GIFs (much better than giftrans)
  • quiet mode (-q) - show only errors/warnings
  • show time taken in m:s
  • clean up output a little
Webpack 0.5b released 13/09/2004
  • catch (more) pngcrush failures
  • minor cleanups
  • more sanity checking
  • implement process-existing-files-only-if-source-newer-than-target (-u) - now can use in an automated way to keep a website updated
  • implement force overwrite (-f) (hidden for the moment)
Webpack 0.6b 4/10/2004
  • WAY more sanity checking
    • check if source or destination is the root directory
    • check if output is same as, subdirectory of or parent of current directory
    • now checks all stupid combinations of source/destination directories I could imagine
  • enable force overwrite (-f) option
  • works with (more) paths with spaces in them
  • add support for HTML::Clean for html compression
  • add support for advancecomp (compression optimizer):
    • use advdef to recompress *.gz (and *.png if pngcrush not installed)
    • use advzip to recompress *.zip
    • brute mode (-b) enables -4 (compress extreme) option on advdef/advzip

To-do

This is a rough list of things to implement or that have been suggested from users.
  • Debug/verbose mode (print input & output files, directory creation etc)
  • Find/create a better html compressor (better than htmlcrunch)
  • Perform duplicate checking of output dir & inform user of duplicates
Let us know if you find this useful,l and if you have suggestions/patches/praise/bugreports for it as they are most welcome.