The "PG2K" Java/servlet "dynamic" implementation of the DHD Multimedia Gallery.


Packages AI code for predicting the popularity or "score" of still images. Package of test/bogus Scorer classes (usually non-parameterised) against which the real parameterised Scorers are measured for efficacy.  
org.hd.d.pg2k.clApp.atHome Application to support remote donation of CPU time to Gallery-related projects in the style of SETI@home.
org.hd.d.pg2k.clApp.offline Off-line utilities designed to help administer the Gallery, without the run-time worries of security and memory and CPU limits, for example: To create/cache pre-computed data (exhibit properties) for faster start-up.
org.hd.d.pg2k.clApp.uploader Code for the Gallery JWS (Java Web Start) exhibit uploader.
org.hd.d.pg2k.svrCore Core classes needed by the entire PG2K server, any EJBs, client applications, WAR, etc, and that do not depend on any PG2K classes ourside this svrCore.
org.hd.d.pg2k.svrCore.datasource Data about exhibits is brought into the system via classes implementing SimpleExhibitPipelineIF.
org.hd.d.pg2k.svrCore.datasource.simpleCache In-pipeline (simple) cache mechanism.
org.hd.d.pg2k.svrCore.FEC Contains Forward Error Correction (FEC) implementation used to protect/repair raw exhibit data.
org.hd.d.pg2k.svrCore.location Data on exhibit location and date/time, and other geographically-related facilities.
org.hd.d.pg2k.svrCore.mediahandler Package for classes to handle creation of thumbnails and other transformations and processing of exhibits, selected by type from the (normalised) exhibit file extension.
org.hd.d.pg2k.svrCore.MIME MIME types and handlers for exhibits.
org.hd.d.pg2k.svrCore.props Loadable (and dynamically reloadable) properties for PG2K, including GenProps which is passed from the master server to any slaves and regulates many general aspects of system behaviour, and LocalProps which contains properties for the local JVM only such as local disc space available.
org.hd.d.pg2k.svrCore.stats Package to handle the collection and display of simple statistics.
org.hd.d.pg2k.svrCore.uploader Common support code for exhibit upload to the Gallery.
org.hd.d.pg2k.svrCore.vars Describes system variable types, values, and the legitimate extant set of such (distributed) variables in the system. Development-only test classes (primarily JUnit). Deals with advertisements, eg helping with their selection and insertion into JSP pages.
org.hd.d.pg2k.webSvr.bgImg Generation of manufactured background images, eg for the default skin.
org.hd.d.pg2k.webSvr.catalogue Explicit support for catalogue pages and trees, eg for HTML/JSP search pages, and pagination of large virtual pages.
org.hd.d.pg2k.webSvr.exhibit Direct support for manipulation of exhibit data, and its serving to users.
org.hd.d.pg2k.webSvr.location WAR/Web support for Location features.
org.hd.d.pg2k.webSvr.threeD Support for JWS (Java WebStart) 3D Gallery Walkthrough.
org.hd.d.pg2k.webSvr.upload Support for Web-based exhibit (eg image) upload, including server side of JWS (Java WebStart) version.
org.hd.d.pg2k.webSvr.util Utilities to support construction of Web pages, monitor throughput, handle forms, etc.
org.hd.d.pg2k.webSvr.virtualHosts Support for virtual hosts, ie supporting multiple URLs with possibly-different livery and roots, from the main Web server.
org.hd.d.pg2k.webSvr.virtualHosts.AlohaEarth Supports Aloha Earth virtual site.


The "PG2K" Java/servlet "dynamic" implementation of the DHD Multimedia Gallery.

The DHD Multimedia Gallery exists as a "pro bono" source of multimedia for people who cannot produce their own or who cannot use a commercial provider. The Gallery started (as the DHD Photo Gallery) when digital cameras and scanners were relatively expensive, and so was a rare source of free/cheap digital Web graphics.

The Gallery now has increasingly varied and comprehensive company in the genuinely-cheap/free multimedia area, which is very good news from everyone from schoolkids to Web masters!

PG2K replaces (and in some cases steals the code from) an earlier incarnation (well two: one based on shell scripts and ImageMagick, and a later variant that replaced most of the scripts with Java and wrote the entire Gallery out as a series of static HTML files (and pre-generated thumbnails, etc) so that it could be served by an ordinary Apache Web server with no CGI). The earlier "static" mechanism was very efficient and safe to serve but was starting to take more than 24 hours to rebuild, even incrementally, and had limited flexibility. It was also repeatedly overrun by greedy spiders/robots opening tens of concurrent connections each and bringing down the whole system and my expensive Internet link.

The servlet version of the Gallery has allowed the distribution to live replicas geographically distributed (in the US, UK, AU and IN as at end-2009), running on fairly cheap Linux, Solaris and ARM boxes from circa sub-1GIPS/1GB capacity up to multi-CPU/multi-GB/multi-GHz boxes.

PG2K uses some live geo-sensitive load-balancing, has some i18n support, and a few interesting projects cooking (see the to-do list.)

With JDK 1.5+, heavy use has been made of the the new JSR-166 concurrency utilities to improve throughput, especially RentrantLock and tryLock() and the reader/writer lock in the ExhibitDataSimpleCache, and the Semaphore in ExhibitDataHTTPTunnelSource. Also, use has been made of the improved semantics of volatile, and of the AtomicXXX types, to convert some code to being lockless. An advantage of lockless synchronisation is that it may be more efficient, but in any case it is deadlock-free!

Much effort as at the start of 2006 had been put into improving concurrency to take advantage of the many cores available in the Gallery's new T1000/Niagara/CoolThreads server, and to reduce page latency for users.

At the turn of 2008 much effort has been put into making this Gallery run well in power- (and thus CPU- and IO-) constrained circumstances, so that the master server can run on a ~20W laptop partially powered by renewable energy, and Gallery slaves can be set to reduce power consumption at times of local grid peak demand.

From mid-to-end 2009 further effort was put into reducing memory footprint to allow the possibility of running on a ~5W 'embedded' ARMv5 Linux 1.2GHz/512MB "SheevaPlug" system at the low-end, while still making good use of available resources on the larger hosts. A great deal of attention has had to be paid to trimming data structure footprint, releasing memory when unused (and in an emergency), efficiency and lazy evaluation, and in general adapting memory and CPU use dynamically to fit available resources. The net result is that the SheevaPlug master can run on under 4W from off-grid solar power while also supporting other services such as email, public NTP, and other Web sites.

DHD Multimedia Gallery V1.60.69

Copyright (c) 1996-2012, Damon Hart-Davis. All rights reserved.