dlx.txt

Proposed Expandable Dynamic Link Library specifications:
(revised 6/12/96 to include 16x16 icons and source info)
(revised 8/20/97 to include non-4-bit color icons)

Integrated Graphics
312 Nevada St.
Northfield, MN 55057
hansonr@stolaf.edu

SUMMARY
-------

This DLX specification allows for icons to be stored in dynamic
link libraries which can be expanded to fit, read by the Program
Manager, and targeted by LOADICON or EXTRACTION functions within
the Windows Application Programming Interface. They are really
just simplified DLL files with a certain convenient structure.
The essentials, once again, are as follows:

  1)   Resident Name Table must immediately follow Resource Table
       and it must contain only one entry, "EXPNDABL".
  2)   Resource Table entries RT_GROUP and RT_IMAGE must be in that
       order and be last in the table if others appear.
  3)   Entry Table must follow Resident Name Table, with space
       preferably allowed for expansion of at least a few icons.
  4)   Only resource data that can be moved (i.e. referenced only
       by the Resouce Table) should follow the Entry Table. 

All this really means is that if the library contains any other
information other than resource data, that information must
reside between the New Executable Header and the Resource Table.
The 32-byte alignment factor means that offsets up to 32*2^16-32
= 2,097,120 may be referenced (about 2 Mb). At 824 bytes per icon
(800 bytes of aligned data + 24 bytes of unaligned Resource Table
space) less overhead for the various DLL tables (64+64+6+12+2=148
bytes), this allows for 2544 icons to be stored in a single DLX
file. 

Perhaps there are slicker, already defined, ways of expanding a
DLL file, but I don't know of any. Then again, I don't know A
LOT! Give me a buzz (hansonr@stolaf.edu) if you know better how
to do this. 

IMAGES

Image data pointed to by the RT_IMAGE name entries
must conform to standard icon structure, consisting of
a 40-byte header along with:


dimension   color mode  color map   XOR bitmap    AND mask     Total bytes
                         (bytes)     (bytes)       (bytes)    (with header)

 16x16        1-bit          8           64           64         176
 16x16        4-bit         64          128           64         296
 16x16        8-bit       1024          256           64        1384
 16x16       16-bit          0          512           64         616
 16x16       24-bit          0          768           64         872

 32x32        1-bit          8          128          128         304
 32x32        4-bit         64          512          128         744
 32x32        8-bit       1024         1024          128        2216
 32x32       16-bit          0         2048          128        2216
 32x32       24-bit          0         3072          128        3240

After each image is a set of 24 informational bytes which give more 
information relating to this image:

info bytes 0-7 (8 bytes)

In the original implementation, 16x16 icons were stored with inadequate
treatment of the AND bytes. This was fixed in version 4.1. 

In the latest implementation of the DLX, these 24 bytes are
used for special purposes. The first 8 of these 24 bytes is an ASCII
indication of what kind of icon was ORIGINALLY placed in the library. 

  "=16x16  "  means a 16x16 4-bit icon stored as such in a 320-byte slot
  "=16x16x2"  means a 16x16 4-bit icon stored in 32x32 mode by making each
              pixel a 2x2 block of equivalent pixels.(abandoned)
  "=16x16++"  means a 16x16 4-bit icon stored as such, but in a larger space;
              This shows up if a 16x16 icon is moved into a 32x32 or 
              16x16x2 slot.            
  "=32x32  "  means a 32x32 4-bit icon stored in an 768-byte slot.
  "=32x32++"  means a 32x32 4-bit icon stored in a larger slot.

  "=nnxnn_b"  where 'nn' is '16' or '32' and b is '1', '4', '8', '0'(16) or '2'(24). 
              Slot size is determined by the type of icon (see above) 

Any other value in this field defaults to "=32x32  "

info bytes 8-19 (12 bytes)

Next comes the source file name field. Since this icon is in a 
library, it comes from somewhere. This field is a DOS 8+3 file
name in ASCII. It is passed on if the icon is transfered to 
another DLX file. 

info bytes 20-23 (4 bytes)

Finally, comes the actual size of the slot. This is a data 
structure consisting of "==" followed by an integer, the value
of which represents the total slot size, in bytes. It is only
significant if the identifier string has in it "++". In addition,
if the "==" is not present, the slot size is assumed to be 768 bytes,
the slot size for a standard 32x32 icon.

Each icon image is padded with extra bytes to bring it up to the required 
32-byte boundary:

 16x16_1        176 + 24 =   200 + 24 =   224 (7 blocks)
 16x16_4        296 + 24 =   320              (10 blocks)
 16x16_8       1384 + 24 =   1408             (44 blocks)
 16x16_0(16)    616 + 24 =   640              (20 blocks)
 16x16_2(24)    872 + 24 =   896              (28 blocks)

 32x32_1        304 + 24 =   328 + 24 =   352 (11 blocks)
 32x32_4        744 + 24 =   768              (24 blocks)
 32x32_8       2216 + 24 =   2248             (70 blocks)
 32x32_0(16)   2216 + 24 =   2248             (70 blocks)
 32x32_2(24)   3240 + 24 =   3264             (102 blocks)


Thus, along with the 22-byte header in each case (also padded to 32 
bytes, a standard 16x16 4-bit icon requires 352 bytes; a standard 
32x32 4-bit icon requires 800. 

An actual 32x32 4-bit image entry reads as follows:

0-3          00000028     size of this header (40 bytes)
4-7          00000020     width of image (32 for icon)
8-11         00000040     height of image (64, because it includes an
                          AND mask; for a BMP file, this is 00000020
12-13        0001         number of planes (must be 1)
14-15        0004         bits/pixel (1 for monochrome, 4 for 16-color,
                          8 for 256-color, 16 for 32-K color, and 
                          24 for 24-bit true color. These last two are NOT
                          supported for storage in DLX files at this time.)
16-19        00000000     compression type; 0 for icons
20-23        00000280     size of image, 640 bytes
24-27        00000000     designed X pixels per meter
28-31        00000000     designed Y pixels per meter
32-35        00000000     0 or actual length of table space in file
36-39        00000000     0 or number of colors important
40-103       whatever     color table (64 bytes, four bytes per entry)
104-615      whatever     XOR bitmap, starting with the last line of
                          the table, indicating (usually) the 4-bit or
                          8-bit pointer index to the color table
616-743      whatever     AND bitmask
744-751      "=nnxnn  "   icon type
752-763      filename.ext originating file
764-765      "=="
766-767      0300         slot size

Note that 744 + 22 (for the icon header) gives 766, the size of a
standard icon file. Indeed, an ICO file is just the above 22-byte
icon header (without the padding) followed immediately by the
bitmap header, color table, XOR bitmap, and AND mask. In the case
of ICO files, the image data pointer at bytes 18-21 reads
"00000016", indicating that the image data follows the icon
header immediately at file offset 22. 

An actual 16x16 image entry reads as follows:

0-3          00000028     size of this header (40 bytes)
4-7          00000010     width of image (32 for icon)
8-11         00000020     height of image (64, because it includes an
                          AND mask; for a BMP file, this is 00000020
12-13        0001         number of planes (must be 1)
14-15        0004         bits/pixel (1 for monochrome, 4 for 16-color,
                          8 for 256-color, 24 for 24-bit true color.)
16-19        00000000     compression type; 0 for icons
20-23        000000A0     size of image, 160 bytes                 
24-27        00000000     designed X pixels per meter
28-31        00000000     designed Y pixels per meter
32-35        00000000     0 or actual length of table space in file
36-39        00000000     0 or number of colors important
40-103       whatever     color table (64 bytes, four bytes per entry)
104-231      whatever     XOR bitmap, starting with the last line of
                          the table, indicating (usually) the 4-bit or
                          8-bit pointer index to the color table
232-295      whatever     AND bitmask
296-303      "=nnxnn  "   icon type
304-315      filename.ext originating file
316-317      "=="
318-319      0140         slot size  

The function of the AND mask is to indicate which pixels to blank
prior to writing the icon, and which to leave unchanged.
Generally a 0 in the AND mask means "put my color here" and a 1
in the mask means "leave this color alone." Generally a 0 in the
AND mask corresponds to a pixel with color in the XOR bitmap; a 1
in the AND mask best correlates with a request for black,
000000h, that is, to whichever index of the color table hold the
long hex value 00000000. 

I originally misinterpreted the and mask of 16x16 icons. In version
4.1 they are properly handled, and even Windows 3.1 likes them.
It turns out that for 16x16 icons, the AND mask, 32 bytes, is stored 
in 64 bytes in the lower half of each of 16 32-bit words. 

Note that the AND mask, though just a string of bits such as
0001001000110100.... must be laid down carefully, 
considering the way that bits of numbers are laid down on 
a PC, rearranged in 16-bit chunks:

binary:      0001 0010 0011 0100  
hex:           1    2    3    4        

is actually 3412h, not 1234h. Nonetheless, the AND mask DOES read
left-to-right, with no rearranging. 

BMP files, on the other hand, contain only the bitmap header (40
bytes), color table (64 bytes), and the XOR table (512 bytes for
32x32). They have their own special 14-byte header as follows:

32x32 bitmap:

0-1          "BM"         file type indicator
2-5          00000276     total file size (630 bytes for 32x32)
6-7          0000         reserved
8-9          0000         reserved
10-13        00000076     file offset to XOR image

16x16 bitmap:

0-1          "BM"         file type indicator
2-5          000000F6     total file size (246 bytes for 16x16)
6-7          0000         reserved
8-9          0000         reserved
10-13        00000076     file offset to XOR image

Thus, a BMP file corresponding to a 32x32 16-color icon is
14+40+64+512 = 630 bytes, and a BMP file for a 16x16 icon
is 14+40+64+128 = 246 bytes. Note that the AND mask is lost
and must be reconstructed by the user if they are making an icon 
from a bitmap image. This is why in Icon Mover an option exists to make
one color transparent. That color is set to black in the color
table, and all pixels referencing it in the AND mask are set to
1. Without this option, all icons made from bitmaps would appear
with a square background rather than the spiffy "cutout" image
common to icons. In addition, for bitmaps, the image height
field is halved to indicate no presense of an AND mask.

All bitmaps in DLX files are stored as icons, not simple bitmaps.

THE BASIC IDEA
--------------

Expandable DLL files (extension DLX) are dynamic link libraries
that may be extended using a simple procedure at run-time. To be
expandable, the library must consist of at least six sections:

  1)   a 64-byte (minimum) DOS executable header,
  2)   a 64-byte New Executable header,
  3)   a Resource Table containing pairs of entries: RT_GROUP and
       RT_IMAGE (one of each type, 12 bytes each, must be present
       for each icon),
  4)   a 12-byte Resident Name Table referencing the name
       "EXPNDABL",
  5)   a 2-byte empty Entry Table, and
  6)   a section for data referenced by the RT_GROUP and RT_IMAGE
       structures.

All specifications for these sections are described below. The
order is critical and must be as listed above. Any other
information present (executable code, for example) must come
PRIOR to the Resource Table and NOTHING except slack room ready
for expansion (2x12 bytes per icon added) may appear between the
Resource Table and the entry table. 

Basically, when an icon is added to the library, the Resource
Table must expand by adding one new RT_GROUP name entry and one
new RT_IMAGE group entry. The Resident Name Table is used as a
reference to the end of the Resource Table; the Entry Table marks
the end of the free space in the file. Thus, two pointers, one to
the Resident Name Table, and one to the Entry Table, delineate
exactly how much room is available for easy addition of icons.
After that space is filled, some data pointed to by the Resource
Table needs to be moved to the end of the file.  

what happens is that icons are added by:

 a)    appending the ICON data to the file (800 bytes, see below);

 b)    extending the Resource Table by 24 bytes, making room for
       one new RT_GROUP and one new RT_IMAGE structure;

 c)    rewriting the Resident Name Table just after the newly
       extended Resource Table; and 

 d)    updating the New Executable Header Resident Name Table
       pointer.

The RT_GROUP structures come first. They take the form of Name
Entries, each of which is 12 bytes and are described in detail
below. As icons are added, the Resident Name Table pushes toward
the Entry Table. Inspection of the New Executable Header allows a
determination of how much room is available. If not enough room
is present, then the following steps are taken:

 a)    the first icon (800 bytes total) after the Entry Table is
       popped to the end of the line;
 b)    the Entry Table (0000h) is rewritten just before the SECOND
       (now first) icon;
 c)    the New Executable Header is revised to indicate the new
       Entry Table offset; and
 d)    the Resource Table is looked through, and the appropriate
       RT_GROUP and its corresponding RT_IMAGE offset references
       are revised to point to the new ICON data positions in the
       file;
 
That's a bit of work, but really it goes very fast, and since
each icon is 800 bytes, that allows for 33 more icon entries in
the Resource Table (2x12 bytes each) before the next icon needs
to be moved. IconMover does it slightly more efficiently,
calculating for a collection of icons how much room is needed,
and then doing it all at once prior to adding any icons. Moving
800 bytes to the end of a file on any 80x86 machine using GET and
PUT takes essentially no time whatsoever.

HOW ICON MOVER DOES IT
----------------------

The exact steps taken by Icon Mover are as follows:

 a)    determine the icons to be appended and append them, being
       careful to align all data along 32-byte boundaries due the
       32-byte blocking factor;
 b)    concurrently write pairs of RT_GROUP and RT_IMAGE NameEntry
       structures to a temporary file. (12 bytes each);
 c)    do the calculation and move whatever icon data need to be
       moved to the end of the file;
 d)    look through the Resource Table for data entries in the
       region that was moved and revise those entries;
 e)    pad out the Resource Table to make room for the new entries;
 f)    add the new entries from the temporary file; and, finally,
 g)    rewrite the Resident Name Table (12 bytes) and the Entry
       Table (2 bytes) and update the New Executable Header to
       indicate their new offsets.

The key is to this arrangement is simply that the DLL file
conform to the above specifications. Not all DLL files do,
because there is no specific requirement for the ordering of
structures in a DLL file. Some DLL files are considerably more
sophisticated, of course, and must include several other
structures. In principle, even these DLL files could be made
expandable by careful rearranging of their component structures.
Icon Mover does not attempt that, however, as I learned all this
empirically with the aid of PC MAGAZINE's ICONJACK program for
reference (6/27/95 issue). 

PROGRAMING DETAILS
------------------

HEADERS

It turns out that the headers can be considerably simpler than
generally described. No one really needs a DLL file to say, "This
program requires Microsoft Windows," for example. Thus, much of
the header information can be ignored (left 00h). As far as I can
tell, only the following aspects of the headers are strictly
required:

offset(dec)         contents     description

0-63                             minimal  64-byte DOS header

       0-1          "MZ"         DOS executable header signature
       60-61        0040         pointer to new header (byte 64 in this
                                 case)

64-127                           minimal 64-byte "New Executable" header

       64-65        "NE"         New Executable header signature
       68-69        *            Entry Table Absolute Offset
       70-71        0002         Entry Table length (indicating just 2
                                 bytes)
       100-101      0040         Resource Table Relative Offset
       102-103      *            Resident Name Table Relative Offset
       118-119      0208         Necessary under Windows '95 only
       126-127      030A         Windows Version expected

where contents are in hex and * means an integer (2-byte) pointer
to an address offest in the file (first byte is 0 in this
system). Only the Entry Table offset and the Resident Name Table
offset change as icons are added to the library.

Resource Table

The Resource Table consists of a 2-byte shift count followed by
blocks of TYPEENTRY and NAMEENTRY entries, followed by an
ENDOFTABLE mark:

       SHIFTCOUNT
       TYPEENTRY1
         NAMEENTRY1a
         NAMEENTRY1b
         NAMEENTRY1c
         etc...
       TYPEENTRY2
         NAMEENTRY2a
         NAMEENTRY2b
         NAMEENTRY2c
         etc...
       etc...
       ENDOFTABLE

SHIFTCOUNT is an integer n, where 2^n is the "alignment factor"
by which all offsets in the Resource Table must be multiplied to
get the absolute file offset of an icon directory or image. In my
implementation, this is 05h, indicating that all data are aligned
along a 32-byte boundary. Thus, an offset of "10h" is really at
file offset 16x32=512 (200h). ENDOFTABLE is a two-byte marker,
0000h. Each entry is a structure of 8 bytes (TYPEENTRY) or 12
bytes (NAMEENTRY):

Type TYPEENTRY 
 id As Integer            RT_GROUP = 800Eh, RT_IMAGE = 8003h
 count As Integer         how many name entries
 reserved As Long         000000000h
End Type

Type NAMEENTRY
 offset As Integer        from beginning of file, NOT relative to the
                          NEH, this is the true file offset divided by
                          the alignment factor, 32 in my
                          implementation. 
 length As Integer        32 for an icon directory entry (RT_GROUP); 
                          768 for a bitmap image entry (RT_IMAGE)             
 Flags As Integer         1C30h for RT_GROUP; 1C10h for RT_IMAGE
 id As Integer            8000h + (index of this name entry, starting
                          with 1)
 handle As Integer        0000h        
 usage As Integer         0000h
End Type

In my version of the DLX file, this table starts at 80h (offset
128). It's pointer in the file at byte 100 (NEH offset + 36). In
my DLX files, this reads "40h", reflecting that this pointer is
relative to the beginning of the New Executable Header, 40h(40h +
40h = 80h). Any additional segments, data, or otherwise may
appear in the DLX file between the New Executable Header and the
Resource Table, however none are needed. In addition, the
RT_GROUP and RT_IMAGE entries should be the last in the table if
other entries exist. 


RESIDENT NAME TABLE

The Resident Name Table is a single 12-byte entry indicating that
this is an expandable DLL file:

  0          08h          length of name, eight bytes
  1-8        "EXPNDABL"
  9-11       00-00h       end-of-name and end-of-table marks

A pointer to this table must be in the New Executable Header at
file byte 102 (NEH offset + 38). It, like the pointer to the
Resource Table, is an offset relative to the beginning of the New
Executable Header. In these DLX specifications, the Resident Name
Table must IMMEDIATELY follow the Resource Table, and there must
be only unused storage space after it.


ENTRY TABLE

There is no true entry point for a DLX file. This table is simply
two bytes, 0000h, situated just prior to the icon resources
themselves (the directories and images pointed to by the NAMETYPE
entries of the Resource Table). Thus, the Entry Table simply
marks the beginning of the data which may be pushed to the end of
the file when more room for the Resource Table is needed. An
integer pointer to this table must be present at byte 68 of the
file (NEH offset + 4). This pointer is an ABSOLUTE offset, not
relative to the New Executable Header.


ICON DIRECTORY

The RT_GROUP name entries point to individual Icon Directories,
which themselves are 22 bytes, padded to 32 bytes by 00h to align
them along a 32-byte boundary. These directories are essentially
what is at the beginning of all ICO files:

0-1          0000         reserved
2-3          0001         type, must be 1
4-5          0001         count, must be 1 for DLX; could be more in
                          other DLL files

6            20           width of image (32 bytes = 20h)
7            20           height of image
8            10           number of colors (repeated in bitmap data)
9            00           reserved
10-11        0001         number of planes (1; all icon bitmaps are
                          single-plane)
12-13        0004         number of bits/pixel; this depends upon the
                          number of colors, and is repeated in the
                          bitmap data
14-17        00000sss     size of the image (0x0280 for 32x32; 0x0A0 for 16x16)
18-21        00000nnn     POINTER TO RESOURCE RT_IMAGE ENTRY; nn = 01,
                          02, etc... 
22-31        00-00        padding to fill 32-bit boundary.


