
                  OS/2 Programming                 (Fidonet)

                 Saturday, 23-Oct-1999 to Friday, 29-Oct-1999

+----------------------------------------------------------------------------+

From: MIKE RUSKAI                                       23-Oct-99 08:17:00
  To: JONATHAN DE BOYNE POLLARD                         23-Oct-99 20:30:10
Subj: Attn: HPFS gurus

Some senseless babbling from Jonathan De Boyne Pollard to Mike Ruskai
on 10-20-99  08:58 about Attn: HPFS gurus...

 MR> Which is more likely, that in rare cases the above would not read LSN
 MR> 16 at all, or that in rare cases the SuperBlock doesn't reside at LSN
 MR> 16 at all?

 JDBP> The former.

 JDBP> As I said before, when one opens a volume for reading using category
 JDBP> 0x08 DosDevIOCtl, logical sector 0 is in fact the start of the "drive"
 JDBP> that contains the volume.  This sector contains the MBR for that
 JDBP> "drive".   
 JDBP> For primary partitions, this "drive" is the actual physical drive and
 JDBP> the MBR is the primary MBR.  The first primary partition usually starts
 JDBP> on the next track boundary (because FDISK utilities try to align
 JDBP> partitions on track boundaries).  This was the cause of your previous
 JDBP> problem. 
[snip]

This explains quite a bit.  I was looking for a somewhat simpler method
(which worked for apparently all logical drives), so it's fortunate that
Vitus posted the information he did, which I'm now using.  Putting the
drive in sector mode overcomes the limitations, while freeing me from
figuring out where the drive actually begins.  Suitable simplicitly for the
simple nature of what I'm trying to do.

I'll also point out at this point that I received the information I needed
about HPFS's freespace bitmaps from the author of DFSEE, so the program I
wrote to check them should work on any drive size.

It can be downloaded from:

http://home.att.net/~thanny/fspace.zip

The source is included, of course.  On all of my drives, I get a 4096
sector discrepancy (2MB) between what DosQueryFSInfo() returns, and what
the freespace bitmaps themselves say about the number of free sectors on
the volume.

Mike Ruskai
thannymeister@yahoo.com


... 'If it ain't broke, you can probably still fix it.' - Tim Allen

___ Blue Wave/QWK v2.20
--- Platinum Xpress/Win/Wildcat5! v3.0pr2
 * Origin: FIDO QWK MAIL & MORE!  WWW.DOCSPLACE.ORG (1:3603/140)

+----------------------------------------------------------------------------+

From: MIKE RUSKAI                                       23-Oct-99 08:22:00
  To: DARIN MCBRIDE                                     23-Oct-99 20:30:11
Subj: HPFS freespace bmp list

Some senseless babbling from Darin Mcbride to Mike Ruskai
on 10-22-99  08:17 about HPFS freespace bmp list...

 MR> What I'd like to do now is distribute the program that does this
 MR> comparison, to get a wider range of data.  Maybe HPFS386 doesn't have
this
 MR> problem, or a different version of HPFS, etc.
 
 MR> The problem is that I'm not certain how to handle drives larger than 4GB.
 MR> The freespace bitmap list is four sectors long, which allows for 512 data
 MR> bands (about 4GB).  Also in the HPFS SuperBlock is a pointer to a spare
 MR> freespace bitmap list.  What's not there is the length of this spare
list.
 MR> If it's only four sectors, like the primary list, then HPFS dies at 8GB,
 MR> without another list.
 
 MR> Unfortunately, I don't have any HPFS partitions larger than 4GB that I
can
 MR> poke around on.

 DM> Send me the source (dmcbride@tower.to.org), and I'll send you back its
 DM> output on my 7.99 GB HPFS drive. (8189MB)

 DM> The reason I ask for source is so I can convince myself you're not
 DM> going to damage my harddrive.  ;-)  Also, I assume you're using VAC 3.0
 DM> or gcc (EMX 0.9d) to compile it.

I wouldn't dream of distributing a "low-level" hard drive program without
source ;)

As it turns out, I found out what should make it work on drives of any
size.  You can download the program at:

http://home.att.net/~thanny/fspace.zip

That includes the source code, and a VACPP 3.08-compiled executable.

The output should look like this (from my 3GB drive):

Reported free sectors: 2956682

Actual free sectors: 2960778

Free sector discrepancy: 4096

Reported free space: 1513821184

Actual free space: 1515918336

Free space discrepancy: 2097152

I haven't done any work towards figuring out where those free but
unreachable sectors are.  I might do that when I'm done writing this, since
I have an empty partition to fill up just for the purpose :)

Mike Ruskai
thannymeister@yahoo.com


... Bones, examine the yeast creature. He's bread, Jim.

___ Blue Wave/QWK v2.20
--- Platinum Xpress/Win/Wildcat5! v3.0pr2
 * Origin: FIDO QWK MAIL & MORE!  WWW.DOCSPLACE.ORG (1:3603/140)

+----------------------------------------------------------------------------+

From: Jonathan de Boyne Pollard                         22-Oct-99 08:20:06
  To: MIKE RUSKAI                                       24-Oct-99 10:30:07
Subj: HPFS freespace bmp list

 MR> What I did next was read all the freespace bitmaps (programmatically,
 MR> of course), and add up the free sector count that way.  

Did you remember to include the bitmap for the directory band ?  It's not
included in the bitmap list, and is listed separately in the super block.

  JdeBP 

--- FleetStreet 1.22 NR
 * Origin: JdeBP's point, using Squish <yuk!> (2:257/609.3)

+----------------------------------------------------------------------------+

From: Jonathan de Boyne Pollard                         22-Oct-99 08:35:16
  To: MIKE RUSKAI                                       24-Oct-99 10:30:07
Subj: HPFS freespace bmp list

 MR> The problem is that I'm not certain how to handle drives larger than
 MR> 4GB. The freespace bitmap list is four sectors long, which allows for
 MR> 512 data bands (about 4GB).  Also in the HPFS SuperBlock is a pointer
 MR> to a spare freespace bitmap list.  What's not there is the length of
 MR> this spare list. If it's only four sectors, like the primary list,
 MR> then HPFS dies at 8GB, without another list.

The length of the bitmap list is proportional to the drive size.  On my
7.82GiB HPFS volume the bitmap list is 8 sectors long, and contains pointers
to 1024 bitmaps (of which 1002 are used).  My educated guess is that the
length of the bitmap list is calculated by the HPFS driver from the overall
volume size.

The spare list sector in the superblock is zero, by the way.

  JdeBP 

--- FleetStreet 1.22 NR
 * Origin: JdeBP's point, using Squish <yuk!> (2:257/609.3)

+----------------------------------------------------------------------------+

From: Darin McBride                                     24-Oct-99 13:18:29
  To: Mike Ruskai                                       24-Oct-99 22:12:00
Subj: HPFS freespace bmp list

 MR> I wouldn't dream of distributing a "low-level" hard drive program without
 MR> source ;)

:-)

[0] f:\new\d>fspace.exe f:

Reported free sectors: 13506154

Actual free sectors: 13510250

Free sector discrepancy: 4096

Reported free space: 6915150848

Actual free space: 6917248000

Free space discrepancy: 2097152

[0] f:\new\d>dspace_d f
f: 6595 of 8189 MB (80.53% free)

[0] f:\new\d>alias dspace_d
echo %1: %@diskfree[%1,M] of %@disktotal[%1,M] MB
(%@eval[%@diskfree[%1,b]/%@disktotal[%1,b]*100=.2]%% free)


---
 * Origin: Tanktalus' Tower BBS (1:250/102)
292/854

+----------------------------------------------------------------------------+

From: MIKE RUSKAI                                       24-Oct-99 11:41:00
  To: JONATHAN DE BOYNE POLLARD                         25-Oct-99 05:23:23
Subj: HPFS freespace bmp list

Some senseless babbling from Jonathan De Boyne Pollard to Mike Ruskai
on 10-22-99  08:20 about HPFS freespace bmp list...

 MR> What I did next was read all the freespace bitmaps (programmatically,
 MR> of course), and add up the free sector count that way.  

 JDBP> Did you remember to include the bitmap for the directory band ?  It's
 JDBP> not included in the bitmap list, and is listed separately in the super
 JDBP> block. 

No, I didn't forget it, and I didn't include it.  It doesn't affect
freespace for files (as far as I can tell, that is), which is all that I'm
checking.

Is my assumption correct, in that regard?

Mike Ruskai
thannymeister@yahoo.com


... An optimist is a person without much experience in life.

___ Blue Wave/QWK v2.20
--- Platinum Xpress/Win/Wildcat5! v3.0pr2
 * Origin: FIDO QWK MAIL & MORE!  WWW.DOCSPLACE.ORG (1:3603/140)
292/854

+----------------------------------------------------------------------------+

From: MIKE RUSKAI                                       24-Oct-99 11:44:00
  To: JONATHAN DE BOYNE POLLARD                         25-Oct-99 05:23:23
Subj: HPFS freespace bmp list

Some senseless babbling from Jonathan De Boyne Pollard to Mike Ruskai
on 10-22-99  08:35 about HPFS freespace bmp list...

 MR> The problem is that I'm not certain how to handle drives larger than
 MR> 4GB. The freespace bitmap list is four sectors long, which allows for
 MR> 512 data bands (about 4GB).  Also in the HPFS SuperBlock is a pointer
 MR> to a spare freespace bitmap list.  What's not there is the length of
 MR> this spare list. If it's only four sectors, like the primary list,
 MR> then HPFS dies at 8GB, without another list.

 JDBP> The length of the bitmap list is proportional to the drive size.  On
 JDBP> my 7.82GiB HPFS volume the bitmap list is 8 sectors long, and contains
 JDBP> pointers to 1024 bitmaps (of which 1002 are used).  My educated guess
 JDBP> is that the length of the bitmap list is calculated by the HPFS driver
 JDBP> from the overall volume size. 
 JDBP> The spare list sector in the superblock is zero, by the way.

I recently found that out from the author of a program called DFSEE.  I
guess it was easier for them to just extend the bitmap list than to use the
spare list (which I guess is never used).  The list is apparently expanded
four sectors at a time, according to drive needs.  The following bit of C
arithmetic calculates how many sectors to read:

bmpListSectCount=((((totalSectors>>14)+2)*4)/sectorSize)+1;

The actual count will, of course, be the next multiple of four, but the
above will get all of the bitmap locations.  If an accurate count is
needed, the following (probably overdone) snip will do it, run after the
above:

bmpListSectCount=bmpListSectCount%4==0?bmpListSectCount:(bmpListSectCount/4+1)*
4;

So far, it seems that the 4096-sector discrepancy is universal, except I
haven't yet received results from a HPFS386 system, because it seems
HPFS386 doesn't understand sector mode.

Mike Ruskai
thannymeister@yahoo.com


... Alzheimer's advantage - new friends every day.

___ Blue Wave/QWK v2.20
--- Platinum Xpress/Win/Wildcat5! v3.0pr2
 * Origin: FIDO QWK MAIL & MORE!  WWW.DOCSPLACE.ORG (1:3603/140)
292/854

+----------------------------------------------------------------------------+

From: Vitus Jensen                                      21-Oct-99 20:53:27
  To: MIKE RUSKAI                                       25-Oct-99 05:23:23
Subj: DosDevIOCtl, cat8, f64

Hello Mike,

19.10.99 07:34, MIKE RUSKAI wrote a message to VITUS JENSEN:

...
 VJ>> ========<start>=======================================
 VJ>> DosRead/Write in direct-access mode will normally fail if the
 VJ>> partition is greater than four gigabytes in size.  If you wish
 VJ>> to write an HPFS editor or other tool, you will need to know the
 VJ>> "secret password" that unlocks the big disks.  After you use
 VJ>> DosOpen to get a handle to that volume, use FSCTL FSC_SECTORIO
 VJ>> (0x9014) and in the parameter list, put a pointer to 0xDEADFACE.
 VJ>>  Doing so will put the handle in "sector" mode.  All offsets and
 VJ>> sizes will refer to sectors instead of bytes, allowing you to
 VJ>> address 64Gb.
 VJ>> ========<end>========================================= 
 VJ>> From "Undocumented Features of OS/2" (os2undoc.zip, 48K)

 MR> Interesting.  I've already figured it out using DosDevIOCtl(),
 MR> though the above might prove useful in the future.

 MR> Is that to say that DosSetFilePtr() will also treat parameters as
 MR> sector offsets?

Never used that IOCtl.

I maintain a file/drive/disk editor to help me develop VRAID.  This editor
could switch to sector mode if a disk or drive is larger than 4GB. 
Unfortunately I don't own such a disk and all my test disks combined would
only reach 3*150+500 = 950MB.  So there is no need to do the coding.

I think you have to try it yourself...

C-x C-s
    Vitus

---
 * Origin: Request VRAID - Software RAID for OS/2! (2:2474/424.1)

+----------------------------------------------------------------------------+

From: MIKE RUSKAI                                       24-Oct-99 22:11:00
  To: DARIN MCBRIDE                                     25-Oct-99 17:43:04
Subj: HPFS freespace bmp list

Some senseless babbling from Darin Mcbride to Mike Ruskai
on 10-24-99  13:18 about HPFS freespace bmp list...

 MR> I wouldn't dream of distributing a "low-level" hard drive program without
 MR> source ;)

 DM> :-)

 DM> [0] f:\new\d>fspace.exe f:

 DM> Reported free sectors: 13506154

 DM> Actual free sectors: 13510250

 DM> Free sector discrepancy: 4096

 DM> Reported free space: 6915150848

 DM> Actual free space: 6917248000

 DM> Free space discrepancy: 2097152

[snip]

Well, it seems that the discrepancy is universal.  I found that the sectors
in question were the first 4096 of the last 4100 sectors on the drive (the
last four sectors being the last freespace bitmap on the drive).

I've also found that with at least one version of HPFS386, sector I/O mode
isn't supported.  DosFSCtl() returns an unknown function error.

So, I'll have to stumble around DosDevIOCtl() once again, and see if I
can't properly deal with primary partitions.

Mike Ruskai
thannymeister@yahoo.com


... A cat will always sit on whatever it is you're trying to read or copy.

___ Blue Wave/QWK v2.20
--- Platinum Xpress/Win/Wildcat5! v3.0pr2
 * Origin: FIDO QWK MAIL & MORE!  WWW.DOCSPLACE.ORG (1:3603/140)
292/854

+----------------------------------------------------------------------------+

From: MIKE RUSKAI                                       25-Oct-99 12:15:00
  To: VITUS JENSEN                                      26-Oct-99 11:19:04
Subj: DosDevIOCtl, cat8, f64

Some senseless babbling from Vitus Jensen to Mike Ruskai
on 10-21-99  20:53 about DosDevIOCtl, cat8, f64...

 VJ> Hello Mike,

 VJ> 19.10.99 07:34, MIKE RUSKAI wrote a message to VITUS JENSEN:

 VJ> ...
 VJ>> ========<start>=======================================
 VJ>> DosRead/Write in direct-access mode will normally fail if the
 VJ>> partition is greater than four gigabytes in size.  If you wish
 VJ>> to write an HPFS editor or other tool, you will need to know the
 VJ>> "secret password" that unlocks the big disks.  After you use
 VJ>> DosOpen to get a handle to that volume, use FSCTL FSC_SECTORIO
 VJ>> (0x9014) and in the parameter list, put a pointer to 0xDEADFACE.
 VJ>>  Doing so will put the handle in "sector" mode.  All offsets and
 VJ>> sizes will refer to sectors instead of bytes, allowing you to
 VJ>> address 64Gb.
 VJ>> ========<end>========================================= 
 VJ>> From "Undocumented Features of OS/2" (os2undoc.zip, 48K)
 
 MR> Interesting.  I've already figured it out using DosDevIOCtl(),
 MR> though the above might prove useful in the future.
 
 MR> Is that to say that DosSetFilePtr() will also treat parameters as
 MR> sector offsets?

 VJ> Never used that IOCtl.

 VJ> I maintain a file/drive/disk editor to help me develop VRAID.  This
 VJ> editor could switch to sector mode if a disk or drive is larger than
 VJ> 4GB.  Unfortunately I don't own such a disk and all my test disks
 VJ> combined would only reach 3*150+500 = 950MB.  So there is no need to do
 VJ> the coding. 
 VJ> I think you have to try it yourself...

Fido sure it slow...

I did try it, and it works as stated above.  Seemingly any API that works
with file handles will do it one sector at a time.

The drawback, however, is that HPFS386 doesn't seem to support sector I/O.

Now I have to *really* figure DosDevIOCtl() out (primary partitions don't
conform to the assumptions I made).

Mike Ruskai
thannymeister@yahoo.com


... If at first you don't succeed, you must be using Windows.

___ Blue Wave/QWK v2.20
--- Platinum Xpress/Win/Wildcat5! v3.0pr2
 * Origin: FIDO QWK MAIL & MORE!  WWW.DOCSPLACE.ORG (1:3603/140)
292/854

+----------------------------------------------------------------------------+

From: MIKE RUSKAI                                       25-Oct-99 18:08:00
  To: JONATHAN DE BOYNE POLLARD                         26-Oct-99 11:19:04
Subj: Multiple primary HPFS

I've worked out finding partitions from the partition table with
DosDevIOCtl(), but translating the boot sector location value into a CHS
value.

There's one possibility, however, that I'm not sure how to deal with.

If a person has more than one primary partition on the one drive, all of
which are formatted HPFS (OS/2, and an OS/2 maintenance partition,
perhaps), then I'll be finding several candidate partitions.

Is there a simple way to figure out which one is correct?  Perhaps there's
a value for the partition status indicating whether the partition is
accessible or not?

Mike Ruskai
thannymeister@yahoo.com


... Captain, I sense a million minds staring at my cleavage.

___ Blue Wave/QWK v2.20
--- Platinum Xpress/Win/Wildcat5! v3.0pr2
 * Origin: FIDO QWK MAIL & MORE!  WWW.DOCSPLACE.ORG (1:3603/140)
292/854

+----------------------------------------------------------------------------+

From: Andrew Grillet                                    22-Oct-99 08:19:22
  To: All                                               28-Oct-99 11:07:21
Subj: Fixpaks

Hi All,

Total horror:

I was running Warp Connect with fixpack 40, on a VESA bus 486
when my VGA card suffered hardware death. 

I ahve more Vesa VGA cards than is good for a normal
human, so I simply swapped it out. Unfortunately, it was
a different brand. This meant attempting a switch back
to VGA. This failed - the scan rate was bizarre.

Obviously time for a re-install.

When I came to reinstall, it was disallowed - "the version
you are attempting to install is a lower revision than the
one installed". 

The only solution was a total wipe of the OS partition.
(There may have been others, but I did not have the time
to look).

Let this be a lesson to programmers everywhere:

Fine, give a warning to users, but NEVER PREVENT
them from taking an action you think is bad - they may
have a good reason you did not think of!

What is the official way to reinstall after applyting a fixpack
after you have deleted the archive/backup? Personally, I
would prefer to BUY (Hint, IBM: this means giving you my
hard earned cash) a manufactur-refreshed version.

Andrew






 
... I used to have a life, but I liked mail-reading so much better.
--- Blue Wave/Maximus
 * Origin: Me/2 <Meet-You> (2:254/259)
292/854

+----------------------------------------------------------------------------+

From: Jonathan de Boyne Pollard                         28-Oct-99 07:23:23
  To: MIKE RUSKAI                                       29-Oct-99 13:33:07
Subj: HPFS freespace bmp list

 MR> No, I didn't forget it, and I didn't include it.  It doesn't affect
 MR> freespace for files (as far as I can tell, that is), which is all
 MR> that I'm checking.
 MR>
 MR> Is my assumption correct, in that regard?

With no access to the source for HPFS.IFS, which is really the only way of
knowing exactly how it calculates its free space figure and in particular
whether *it* includes the free space in the directory band, I have no idea.

  JdeBP 

--- FleetStreet 1.22 NR
 * Origin: JdeBP's point, using Squish <yuk!> (2:257/609.3)
292/854

+----------------------------------------------------------------------------+

From: Jonathan de Boyne Pollard                         28-Oct-99 07:31:28
  To: MIKE RUSKAI                                       29-Oct-99 13:33:07
Subj: HPFS freespace bmp list

 MR> Well, it seems that the discrepancy is universal.  I found that the
 MR> sectors in question were the first 4096 of the last 4100 sectors on
 MR> the drive (the last four sectors being the last freespace bitmap on
 MR> the drive).

How did you find that ?  Did you mark the sectors as used by hand and look to
see whether the free space changed ?

  JdeBP 

--- FleetStreet 1.22 NR
 * Origin: JdeBP's point, using Squish <yuk!> (2:257/609.3)
292/854

+----------------------------------------------------------------------------+

From: Roy J. Tellason                                   28-Oct-99 14:00:22
  To: Andrew Grillet                                    29-Oct-99 13:33:07
Subj: Fixpaks

Andrew Grillet wrote in a message to All:

 AG> Hi All,

 AG> Total horror:

 AG> I was running Warp Connect with fixpack 40, on a VESA bus 486 
 AG> when my VGA card suffered hardware death. 

 AG> I ahve more Vesa VGA cards than is good for a normal human, so 
 AG> I simply swapped it out. Unfortunately, it was a different 
 AG> brand. This meant attempting a switch back to VGA. This failed 
 AG> - the scan rate was bizarre.

 AG> Obviously time for a re-install.

 AG> When I came to reinstall, it was disallowed - "the version you 
 AG> are attempting to install is a lower revision than the one 
 AG> installed". 

 AG> The only solution was a total wipe of the OS partition. (There 
 AG> may have been others, but I did not have the time to look).

What about hitting alt-F1 (I think?) when you get the "boot blob" and
selecting plain vga as a video mode?

It's in there.

--- 
 * Origin: TANSTAAFL BBS 717-838-8539 (1:270/615)

+----------------------------------------------------------------------------+

+============================================================================+
