
                  OS/2 Programming                 (Fidonet)

                 Saturday, 06-Nov-1999 to Friday, 12-Nov-1999

+----------------------------------------------------------------------------+

From: James Williams                                    03-Nov-99 10:30:00
  To: Jonathan de Boyne Pollar                          07-Nov-99 14:47:09
Subj: Re: Emx/Gnu

-=> Jonathan de Boyne Pollard wrote to James Williams <=-

 JW> I want to install the EMX/GNU compiler. Would someone please let me
 JW> know the various Environment Variables needed in my config.sys.

 JdP> They are all fully described in the READMEs that are
 JdP> supplied in the archives.  Start with emx\doc\readme.doc
 JdP> and then read, as it tells you to, emx\doc\install.doc .

Thank You very much.


... MultiMail, the new multi-platform, multi-format offline reader!
___ MultiMail/OS/2 v0.32

--- Maximus/2 3.01
 * Origin: COMM Port OS/2 juge.com 204.89.247.1 (281) 980-9671 (1:106/2000)
292/854

+----------------------------------------------------------------------------+

From: Kris Steenhaut                                    02-Nov-99 10:02:09
  To: Andrew Grillet                                    07-Nov-99 14:47:09
Subj: Fixpaks

   Dag Andrew,
vrijdag 22 oktober 1999 08.19, Bericht van Andrew Grillet aan All:

 AG> Hi All,

 AG> Total horror:

 AG> I was running Warp Connect with fixpack 40, on a VESA bus 486
 AG> when my VGA card suffered hardware death.

 AG> I ahve more Vesa VGA cards than is good for a normal
 AG> human, so I simply swapped it out. Unfortunately, it was
 AG> a different brand. This meant attempting a switch back
 AG> to VGA. This failed - the scan rate was bizarre.

 AG> Obviously time for a re-install.

Not so, for two reasons:

1 You only have to delete the syslevel.os2 and syslevel.fpk files in the
\OS2\INSTALL directory.

2 You should have had no trouble at all, if, after having replaced the
hardware, you'd had rebooted with an alt-F1 --> F3 operation.

Just the basics. Next time, before action, do ask first.


    Groeten uit Gent,

      Kris

--- GoldED/2 3.0.1  FMail/2 1.48/g
 * Origin: Hersendood punt #11 (2:292/8125.11)
292/854

+----------------------------------------------------------------------------+

From: Harald Pollack                                    02-Nov-99 09:08:09
  To: MIKE RUSKAI                                       07-Nov-99 14:47:09
Subj: HPFS freespace bmp list

Am 24. Oct 99 22:11, schrieb MIKE RUSKAI (1:3603/140)
    an DARIN MCBRIDE:

Servus MIKE!

 MR> Well, it seems that the discrepancy is universal.  I found 
 MR> that the sectors in question were the first 4096 of the last 4100 sectors 

 MR> on the drive (the last four sectors being the last freespace bitmap on
the
 MR>  drive).

Maybe that space is reserved for a 'Reference Partition' on some IBM PS/2
machines. I have made the experience, that creating a reference partition on a 
(already partitioned) drive which has a HPFS partition as its last one, will
work without destroying any data.

Herzliche Gruesse, Harald

-+- Message created on Tuesday November, 02 1999 09:11:03 MEZ
--- WarpEd/2 1.11
 * Origin: LFP Schwechat [OS/2] (2:310/14.59)
292/854

+----------------------------------------------------------------------------+

From: Jonathan de Boyne Pollard                         03-Nov-99 09:49:25
  To: George White                                      07-Nov-99 14:47:09
Subj: PROMPT $I in PMCMD

 JdBP>> Actually, what I need to do is to write a control window that
 JdBP>> can be used to display coloured text, rather than relying on a
 JdBP>> standard MLE control.  I know that these things can be done,
 JdBP>> because programs like ZOC and LiveWire use such controls for
 JdBP>> terminal emulator windows.  Does anyone know how they do it ?
 JdBP>> I've tried rolling my own, but it is *very* slow in repainting
 JdBP>> (all of those calls to GpiCharStringPosAt() take a long time).

 GW> I seem to remember that a way to do this was to hide the window,
 GW> build the text, and then make it visible.

That isn't the solution.  The text is already "built", in an internal logical
screen buffer.  The problem is that the raw GPI call to actually paint the
character cells onto the device context is slow, and that often many
characters, in different combinations of colours, have to be redrawn during a
WM_PAINT event when the window contents are being redisplayed.

Does anyone here in OS2PROG have any suggestions ?

  JdeBP 

--- FleetStreet 1.22 NR
 * Origin: JdeBP's point, using Squish <yuk!> (2:257/609.3)
292/854

+----------------------------------------------------------------------------+

From: Mario Semo                                        02-Nov-99 08:11:07
  To: MIKE RUSKAI                                       07-Nov-99 14:47:09
Subj: DosDevIOCtl()

Hallo MIKE!

Antwort auf eine Message von MIKE RUSKAI an JONATHAN DE BOYNE POLLARD:

 MR> So, I put the drive into sector mode (using what Vitus posted), which
 MR> makes things much simpler.

 MR> After poking around, I've run into a strange situation, which you can
 MR> read about in a post entitled "HPFS freespace bmp list".

Just a note. at the 2nd and 3rd COLOROS2 (1994,1995) (OS2 Develop Conferences) 
the IBM HPFS developer (Doug Azzarito) had sessions with informations on HPFS
low level layouts. (e.g. about the superblock etc).

Maybe you find the conference proceedings (printed sessions) somewhere on the
net?

Conference Manager was Wayne Kovsky.

heres what i have (but i havent verified if the data is still correct).

http://www.colos2.com                # ColoradOS2
http://www.softwaresummit.com  # ColoradoSoftwareSummit (exColoradOS2)

i just have an old CompuServe id from Wayne:

 Wayne Kovsky 76711,1221 

maybe you can reach him via CIS or at 76711.1221@compuserve.com



Servus, Mario!

--- FleetStreet 1.21 PR#2
 * Origin: LC/32 Development Team, http://www.kirchnersoft.com (2:310/14.11)
396/1

+----------------------------------------------------------------------------+

From: Mario Semo                                        02-Nov-99 07:57:01
  To: MIKE RUSKAI                                       07-Nov-99 14:47:09
Subj: HPFS freespace bmp list

Hallo MIKE!

Antwort auf eine Message von MIKE RUSKAI an ALL:

 MR> On a completely empty drive, the difference between the total drive
 MR> size, and the free space (as reported by DosQueryFSInfo()) has a gap
 MR> of 2MB not accounted for by HPFS structures.  What I first did was

just as a side note:

i primary run NT4.03 in the meantime, but once a day i still boot OS2. and i
ruin NT 4.03 with the NT 3.5HPFS drivers.
and i realized sometime, that NT-HPFS and WARP4 HPFS displays 2 different
values for the free space on a drive. the difference is nearly 2MB!
e.g. a dir on NT says 10MB free.
and a dir on OS2 says 8MB free.

 MR> 512 data bands (about 4GB).  Also in the HPFS SuperBlock is a pointer
 MR> to a spare freespace bitmap list.  What's not there is the length of
 MR> this spare list. If it's only four sectors, like the primary list,
 MR> then HPFS dies at 8GB, without another list.

where in the super block should be this pointer?

I have no real knowledge on HPFS structs, but i opened the ColoradOS2
conferences papers and looked at the HPFS Super block struct. i cant find such 
a pointer.

The HPFS Boot Area has such a pointer.

LSN / Size (Sectors) / Description:

00 / 1 / Standard Boot Sector
01 / 0F / HPFS Boot Code
10 / 1 / Super Block
11 / 1 / Spare Block <---- do you  mean this ?

Servus, Mario!

--- FleetStreet 1.21 PR#2
 * Origin: LC/32 Development Team, http://www.kirchnersoft.com (2:310/14.11)
396/1

+----------------------------------------------------------------------------+

From: Jonathan de Boyne Pollard                         04-Nov-99 22:49:02
  To: MIKE RUSKAI                                       07-Nov-99 14:47:09
Subj: Multiple primary HPFS

 MR> If a person has more than one primary partition on the one drive, all
 MR> of which are formatted HPFS (OS/2, and an OS/2 maintenance partition,
 MR> perhaps), then I'll be finding several candidate partitions.
 MR>
 MR> Is there a simple way to figure out which one is correct?  Perhaps
 MR> there's a value for the partition status indicating whether the
 MR> partition is accessible or not?

Partitions are "hidden" by programs such as Boot Manager by changing their
types to an "unrecognised" type.  Usually this is done by just flipping a bit
or two in the type byte.  So the answer is that if you have "hidden"
partitions they will usually have types that, as standard, operating systems
will not recognise as valid partition types.  There will be only one primary
partition in the primary MBR that has a valid partition type (and that isn't
an extended partition), and that will be your candidate partition.

The reason that it works this way is because of the way that most operating
systems look for partitions when assigning drive letters to them.  They use a
very simplistic algorithm.  They stop processing the table in each MBR when
they reach the first (non-0x05/0x0F) partition with a type that they
recognise.  ( If you want to see how OS/2 does it, look at the code for
OS2DASD.DMD on the OS/2 DDK, and look at the Process_Partition function in the 
file DMBPB.C . )

The slight exception to this rule is Windows NT 4.0, which caters for more
than one primary partition having a recognised type in the primary MBR.  This
allows for up to four primary partitions per disk.  But as the prominent
warning displayed by Windows NT's Disk Manager utility when one tries to
create more than one primary partition explains, this is incompatible with
other operating systems.  (Well, *most* other operating systems.  (-:)

I thought that you had already solved this problem, though.  Didn't you say
that the DosDevIOCtl to obtain the BPB gave you in the "reserved sectors"
field the offset from the start of the "drive" to the partition ?  Surely that 
solves this very problem ?

  JdeBP 

--- FleetStreet 1.22 NR
 * Origin: JdeBP's point, using Squish <yuk!> (2:257/609.3)
396/1

+----------------------------------------------------------------------------+

From: MIKE RUSKAI                                       06-Nov-99 07:20:00
  To: HARALD POLLACK                                    07-Nov-99 14:47:09
Subj: HPFS freespace bmp list

Some senseless babbling from Harald Pollack to Mike Ruskai
on 11-02-99  09:08 about HPFS freespace bmp list...

 HP> Am 24. Oct 99 22:11, schrieb MIKE RUSKAI (1:3603/140)
 HP> an DARIN MCBRIDE:

 HP> Servus MIKE!
 
 MR> Well, it seems that the discrepancy is universal.  I found 
 MR> that the sectors in question were the first 4096 of the last 4100 sectors 

 MR> on the drive (the last four sectors being the last freespace bitmap on
the
 MR>  drive).

 HP> Maybe that space is reserved for a 'Reference Partition' on some IBM
 HP> PS/2 machines. I have made the experience, that creating a reference
 HP> partition on a (already partitioned) drive which has a HPFS partition
 HP> as its last one, will work without destroying any data.

That wouldn't work at all.  That space is not necessarily contigous at all.
It can have two freespace bitmaps right in the middle of it, depending on
the drive layout.

Furthermore, on a 10MB HPFS drive, that area is only 443 sectors, which is
not enough for a PS/2 reference partition (which not very many PS/2's
actually have - most use reference diskettes).

Mike Ruskai
thannymeister@yahoo.com


... A QWK? Stomp on it before it escapes!

___ Blue Wave/QWK v2.20
--- Platinum Xpress/Win/Wildcat5! v3.0pr2
 * Origin: FIDO QWK MAIL & MORE!  WWW.DOCSPLACE.ORG (1:3603/140)
396/1

+----------------------------------------------------------------------------+

From: MIKE RUSKAI                                       06-Nov-99 07:24:00
  To: MARIO SEMO                                        07-Nov-99 14:47:09
Subj: DosDevIOCtl()

Some senseless babbling from Mario Semo to Mike Ruskai
on 11-02-99  08:11 about DosDevIOCtl()...

 MS> Hallo MIKE!

 MS> Antwort auf eine Message von MIKE RUSKAI an JONATHAN DE BOYNE POLLARD:
 
 MR> So, I put the drive into sector mode (using what Vitus posted), which
 MR> makes things much simpler.
 
 MR> After poking around, I've run into a strange situation, which you can
 MR> read about in a post entitled "HPFS freespace bmp list".

 MS> Just a note. at the 2nd and 3rd COLOROS2 (1994,1995) (OS2 Develop
 MS> Conferences) the IBM HPFS developer (Doug Azzarito) had sessions with
 MS> informations on HPFS low level layouts. (e.g. about the superblock
 MS> etc). 
 MS> Maybe you find the conference proceedings (printed sessions) somewhere
 MS> on the net?

[snip]

There doesn't seem to be any info on previous conferences on that page.
Regardless, I probably already have the same information from the EDM/2
site, which covered all the file system structures (just didn't mention
unreachable space at the end of the drive).

Mike Ruskai
thannymeister@yahoo.com


... After seeing Windows I realized Bill Gates is an idiot.

___ Blue Wave/QWK v2.20
--- Platinum Xpress/Win/Wildcat5! v3.0pr2
 * Origin: FIDO QWK MAIL & MORE!  WWW.DOCSPLACE.ORG (1:3603/140)
396/1

+----------------------------------------------------------------------------+

From: MIKE RUSKAI                                       06-Nov-99 07:27:00
  To: MARIO SEMO                                        07-Nov-99 14:47:09
Subj: HPFS freespace bmp list

Some senseless babbling from Mario Semo to Mike Ruskai
on 11-02-99  07:57 about HPFS freespace bmp list...

 MS> Hallo MIKE!

 MS> Antwort auf eine Message von MIKE RUSKAI an ALL:
 
 MR> On a completely empty drive, the difference between the total drive
 MR> size, and the free space (as reported by DosQueryFSInfo()) has a gap
 MR> of 2MB not accounted for by HPFS structures.  What I first did was

 MS> just as a side note:

 MS> i primary run NT4.03 in the meantime, but once a day i still boot OS2.
 MS> and i ruin NT 4.03 with the NT 3.5HPFS drivers.
 MS> and i realized sometime, that NT-HPFS and WARP4 HPFS displays 2
 MS> different values for the free space on a drive. the difference is
 MS> nearly 2MB! e.g. a dir on NT says 10MB free.
 MS> and a dir on OS2 says 8MB free.

So, either OS/2's HPFS.IFS has a bug, or that space is reserved for
something (presumably useful), and NT's driver doesn't know about it.

 MR> 512 data bands (about 4GB).  Also in the HPFS SuperBlock is a pointer
 MR> to a spare freespace bitmap list.  What's not there is the length of
 MR> this spare list. If it's only four sectors, like the primary list,
 MR> then HPFS dies at 8GB, without another list.

 MS> where in the super block should be this pointer?

 MS> I have no real knowledge on HPFS structs, but i opened the ColoradOS2
 MS> conferences papers and looked at the HPFS Super block struct. i cant
 MS> find such a pointer.

 MS> The HPFS Boot Area has such a pointer.

 MS> LSN / Size (Sectors) / Description:

 MS> 00 / 1 / Standard Boot Sector
 MS> 01 / 0F / HPFS Boot Code
 MS> 10 / 1 / Super Block
 MS> 11 / 1 / Spare Block <---- do you  mean this ?

No, it's offset 0x1C in the SuperBlock, which is apparently always 0.  The
primary sector list (offset 0x18) is variable in size, expanding 4 sectors
at a time as necessary, as I later found out.

I've since modified the program in question to use DosDevIOCtl() properly.
I've also consolidated a bunch of stuff into a single program, which I'm
still working on.

You can get a report on your freespace discrepancy with the following
program:

http://home.att.net/~thanny/fspace.zip

It includes the source in the archive.

Mike Ruskai
thannymeister@yahoo.com


... Conscience gets a lot of credit that belongs to cold feet.

___ Blue Wave/QWK v2.20
--- Platinum Xpress/Win/Wildcat5! v3.0pr2
 * Origin: FIDO QWK MAIL & MORE!  WWW.DOCSPLACE.ORG (1:3603/140)
396/1

+----------------------------------------------------------------------------+

From: MIKE RUSKAI                                       06-Nov-99 07:33:00
  To: JONATHAN DE BOYNE POLLARD                         07-Nov-99 14:47:09
Subj: Multiple primary HPFS

Some senseless babbling from Jonathan De Boyne Pollard to Mike Ruskai
on 11-04-99  22:49 about Multiple primary HPFS...

 MR> If a person has more than one primary partition on the one drive, all
 MR> of which are formatted HPFS (OS/2, and an OS/2 maintenance partition,
 MR> perhaps), then I'll be finding several candidate partitions.
 MR>
 MR> Is there a simple way to figure out which one is correct?  Perhaps
 MR> there's a value for the partition status indicating whether the
 MR> partition is accessible or not?

 JDBP> Partitions are "hidden" by programs such as Boot Manager by changing
 JDBP> their types to an "unrecognised" type.  Usually this is done by just
 JDBP> flipping a bit or two in the type byte.  So the answer is that if you
 JDBP> have "hidden" partitions they will usually have types that, as
 JDBP> standard, operating systems will not recognise as valid partition
 JDBP> types.  There will be only one primary partition in the primary MBR
 JDBP> that has a valid partition type (and that isn't an extended partition),
 JDBP> and that will be your candidate partition. 
 JDBP> The reason that it works this way is because of the way that most
 JDBP> operating systems look for partitions when assigning drive letters to
 JDBP> them.  They use a very simplistic algorithm.  They stop processing the
 JDBP> table in each MBR when they reach the first (non-0x05/0x0F) partition
 JDBP> with a type that they recognise. ( If you want to see how OS/2 does
 JDBP> it, look at the code for OS2DASD.DMD on the OS/2 DDK, and look at the
 JDBP> Process_Partition function in the file DMBPB.C . ) 
 JDBP> The slight exception to this rule is Windows NT 4.0, which caters for
 JDBP> more than one primary partition having a recognised type in the primary
 JDBP> MBR.  This allows for up to four primary partitions per disk.  But as
 JDBP> the prominent warning displayed by Windows NT's Disk Manager utility
 JDBP> when one tries to create more than one primary partition explains, this
 JDBP> is incompatible with other operating systems.  (Well, *most* other
 JDBP> operating systems.  (-:) 

I wish Fido were faster :)

I had since already come to that conclusion, by managing to create two
primary partitions on another machine, formatted HPFS.  OS/2 sets the type
to 0x17 when hidden, so the one and only 0x07 in the partition table must
be the correct one, whether or not the volume in question is a primary
partition or a logical drive in the extended partition.

 JDBP> I thought that you had already solved this problem, though.  Didn't
 JDBP> you say that the DosDevIOCtl to obtain the BPB gave you in the
 JDBP> "reserved sectors" field the offset from the start of the "drive" to
 JDBP> the partition ?  Surely that solves this very problem ?

Actually, I was referring to the "hidden sectors" field.  As I implemented
it, it did not work.  I was just adding that value to the sector number of
the TrackTable[] entry.  If I used that value to get a proper CHS value, it
might work.

As it is, I've already done the partition table work.  I read in the first
sector of the "logical drive", and if it ends with 0x55AA, I consider bytes
446 through 509 to be the partition table.  From there, the first (and
necessarily only) type 0x07 partition found is what's desired.  Since the
CHS values aren't necessarily useful (consider a logical drive that's
entirely beyond the 1024th cylinder), I just use the boot sector location.
Add that to the LSN, convert to a CHS value, and DosDevIOCtl() gets what
it's supposed to.  It works on every type of partition I have (I tested
finding FAT drives, too).

I've put all the functions I've been playing around with into a single
program now, which will set the HPFS dirty bit, report the free space
discrepancy, report the total system usage, and dump any sector on the
drive.  Once I think of a few more pseudo-useful things to do, I'll put it
up for download.

Then I might try playing around with a HPFS undelete program, which brings
me to a question about something you mentioned previously.

It does stand to reason that any FNode which resides in a sector marked
free is a deleted file, that could potentially be recovered.  What about
FNodes that happen to reside in the reserved but as-yet unused sectors of a
file?  That is, when a file is created with some slack space to grow into,
it's slack space will be marked used, but not written to yet.  Could there
not be an FNode, or even file data, in that slack space which can be
recovered?

Mike Ruskai
thannymeister@yahoo.com


... Arguing logic with a programmer can get you hexed.

___ Blue Wave/QWK v2.20
--- Platinum Xpress/Win/Wildcat5! v3.0pr2
 * Origin: FIDO QWK MAIL & MORE!  WWW.DOCSPLACE.ORG (1:3603/140)
396/1

+----------------------------------------------------------------------------+

From: MIKE RUSKAI                                       06-Nov-99 08:20:00
  To: JONATHAN DE BOYNE POLLARD                         07-Nov-99 14:47:09
Subj: PROMPT $I in PMCMD

Some senseless babbling from Jonathan De Boyne Pollard to George White
on 11-03-99  09:49 about PROMPT $I in PMCMD...

 JdBP>> Actually, what I need to do is to write a control window that
 JdBP>> can be used to display coloured text, rather than relying on a
 JdBP>> standard MLE control.  I know that these things can be done,
 JdBP>> because programs like ZOC and LiveWire use such controls for
 JdBP>> terminal emulator windows.  Does anyone know how they do it ?
 JdBP>> I've tried rolling my own, but it is *very* slow in repainting
 JdBP>> (all of those calls to GpiCharStringPosAt() take a long time).
 
 GW> I seem to remember that a way to do this was to hide the window,
 GW> build the text, and then make it visible.

 JDBP> That isn't the solution.  The text is already "built", in an internal
 JDBP> logical screen buffer.  The problem is that the raw GPI call to
 JDBP> actually paint the character cells onto the device context is slow, and
 JDBP> that often many characters, in different combinations of colours, have
 JDBP> to be redrawn during a WM_PAINT event when the window contents are
 JDBP> being redisplayed. 
 JDBP> Does anyone here in OS2PROG have any suggestions ?

I can't be sure, because I don't have either program installed, but I
thought both ZOC and LiveWire actually used a VIO window for the terminal.

That would mean a bunch of calls to VioSetState() to set the color
registers.  It'd probably be more work, but if that's what ZOC and LiveWire
used, it must be faster.

Unless, of course, you rely on ANSI for colors.  In that case, you'd just
need to put the control codes into the text you write.

I think I recall a comment from the author of either ZOC or LiveWire, years
ago, explaining why flashing ANSI characters instead have a grey
background, because of how VIO ANSI is handled.

Mike Ruskai
thannymeister@yahoo.com


... BBS addiction is a Terminal disease.

___ Blue Wave/QWK v2.20
--- Platinum Xpress/Win/Wildcat5! v3.0pr2
 * Origin: FIDO QWK MAIL & MORE!  WWW.DOCSPLACE.ORG (1:3603/140)
396/1

+----------------------------------------------------------------------------+

From: MIKE RUSKAI                                       09-Nov-99 17:34:28
  To: HARALD POLLACK                                    09-Nov-99 22:40:27
Subj: HPFS freespace bmp list

[This may be a duplicate message.  My Fido host had a glitch]
[that prevented mail from going out for an unspecified      ]
[length of time, so I have to repost recent messages.       ]


Some senseless babbling from Harald Pollack to Mike Ruskai
on 11-02-99  09:08 about HPFS freespace bmp list...

 HP> Am 24. Oct 99 22:11, schrieb MIKE RUSKAI (1:3603/140)
 HP> an DARIN MCBRIDE:

 HP> Servus MIKE!

 MR> Well, it seems that the discrepancy is universal.  I found
 MR> that the sectors in question were the first 4096 of the last 4100
sectors
 MR> on the drive (the last four sectors being the last freespace bitmap
on the
 MR>  drive).

 HP> Maybe that space is reserved for a 'Reference Partition' on some
IBM
 HP> PS/2 machines. I have made the experience, that creating a
reference
 HP> partition on a (already partitioned) drive which has a HPFS
partition
 HP> as its last one, will work without destroying any data.

That wouldn't work at all.  That space is not necessarily contigous at
all.
It can have two freespace bitmaps right in the middle of it, depending
on
the drive layout.

Furthermore, on a 10MB HPFS drive, that area is only 443 sectors, which
is
not enough for a PS/2 reference partition (which not very many PS/2's
actually have - most use reference diskettes).

Mike Ruskai
thannymeister@yahoo.com

... A QWK? Stomp on it before it escapes!

___ Blue Wave/QWK v2.20

--- Platinum Xpress/Win/Wildcat5! v3.0pr2
 * Origin: FIDO QWK MAIL & MORE!  WWW.DOCSPLACE.ORG (1:3603/140)

+----------------------------------------------------------------------------+

From: MIKE RUSKAI                                       09-Nov-99 17:35:09
  To: MARIO SEMO                                        09-Nov-99 22:40:27
Subj: DosDevIOCtl()

[This may be a duplicate message.  My Fido host had a glitch]
[that prevented mail from going out for an unspecified      ]
[length of time, so I have to repost recent messages.       ]


Some senseless babbling from Mario Semo to Mike Ruskai
on 11-02-99  08:11 about DosDevIOCtl()...

 MS> Hallo MIKE!

 MS> Antwort auf eine Message von MIKE RUSKAI an JONATHAN DE BOYNE
POLLARD:

 MR> So, I put the drive into sector mode (using what Vitus posted),
which
 MR> makes things much simpler.

 MR> After poking around, I've run into a strange situation, which you
can
 MR> read about in a post entitled "HPFS freespace bmp list".

 MS> Just a note. at the 2nd and 3rd COLOROS2 (1994,1995) (OS2 Develop
 MS> Conferences) the IBM HPFS developer (Doug Azzarito) had sessions
with
 MS> informations on HPFS low level layouts. (e.g. about the superblock
 MS> etc).
 MS> Maybe you find the conference proceedings (printed sessions)
somewhere
 MS> on the net?

[snip]

There doesn't seem to be any info on previous conferences on that page.
Regardless, I probably already have the same information from the EDM/2
site, which covered all the file system structures (just didn't mention
unreachable space at the end of the drive).

Mike Ruskai
thannymeister@yahoo.com

... After seeing Windows I realized Bill Gates is an idiot.

___ Blue Wave/QWK v2.20
--- Platinum Xpress/Win/Wildcat5! v3.0pr2
 * Origin: FIDO QWK MAIL & MORE!  WWW.DOCSPLACE.ORG (1:3603/140)

+----------------------------------------------------------------------------+

From: MIKE RUSKAI                                       09-Nov-99 17:35:28
  To: MARIO SEMO                                        09-Nov-99 22:40:27
Subj: HPFS freespace bmp list

[This may be a duplicate message.  My Fido host had a glitch]
[that prevented mail from going out for an unspecified      ]
[length of time, so I have to repost recent messages.       ]


Some senseless babbling from Mario Semo to Mike Ruskai
on 11-02-99  07:57 about HPFS freespace bmp list...

 MS> Hallo MIKE!

 MS> Antwort auf eine Message von MIKE RUSKAI an ALL:

 MR> On a completely empty drive, the difference between the total drive
 MR> size, and the free space (as reported by DosQueryFSInfo()) has a
gap
 MR> of 2MB not accounted for by HPFS structures.  What I first did was

 MS> just as a side note:

 MS> i primary run NT4.03 in the meantime, but once a day i still boot
OS2.
 MS> and i ruin NT 4.03 with the NT 3.5HPFS drivers.
 MS> and i realized sometime, that NT-HPFS and WARP4 HPFS displays 2
 MS> different values for the free space on a drive. the difference is
 MS> nearly 2MB! e.g. a dir on NT says 10MB free.
 MS> and a dir on OS2 says 8MB free.

So, either OS/2's HPFS.IFS has a bug, or that space is reserved for
something (presumably useful), and NT's driver doesn't know about it.

 MR> 512 data bands (about 4GB).  Also in the HPFS SuperBlock is a
pointer
 MR> to a spare freespace bitmap list.  What's not there is the length
of
 MR> this spare list. If it's only four sectors, like the primary list,
 MR> then HPFS dies at 8GB, without another list.

 MS> where in the super block should be this pointer?

 MS> I have no real knowledge on HPFS structs, but i opened the
ColoradOS2
 MS> conferences papers and looked at the HPFS Super block struct. i
cant
 MS> find such a pointer.

 MS> The HPFS Boot Area has such a pointer.

 MS> LSN / Size (Sectors) / Description:

 MS> 00 / 1 / Standard Boot Sector
 MS> 01 / 0F / HPFS Boot Code
 MS> 10 / 1 / Super Block
 MS> 11 / 1 / Spare Block <---- do you  mean this ?

No, it's offset 0x1C in the SuperBlock, which is apparently always 0.
The
primary sector list (offset 0x18) is variable in size, expanding 4
sectors
at a time as necessary, as I later found out.

I've since modified the program in question to use DosDevIOCtl()
properly.
I've also consolidated a bunch of stuff into a single program, which I'm
still working on.

You can get a report on your freespace discrepancy with the following
program:

http://home.att.net/~thanny/fspace.zip

It includes the source in the archive.

Mike Ruskai
thannymeister@yahoo.com

... Conscience gets a lot of credit that belongs to cold feet.

___ Blue Wave/QWK v2.20
--- Platinum Xpress/Win/Wildcat5! v3.0pr2
 * Origin: FIDO QWK MAIL & MORE!  WWW.DOCSPLACE.ORG (1:3603/140)

+----------------------------------------------------------------------------+

From: MIKE RUSKAI                                       09-Nov-99 17:36:24
  To: JONATHAN DE BOYNE POLLARD                         09-Nov-99 22:40:27
Subj: Multiple primary HPFS

[This may be a duplicate message.  My Fido host had a glitch]
[that prevented mail from going out for an unspecified      ]
[length of time, so I have to repost recent messages.       ]


Some senseless babbling from Jonathan De Boyne Pollard to Mike Ruskai
on 11-04-99  22:49 about Multiple primary HPFS...

 MR> If a person has more than one primary partition on the one drive,
all
 MR> of which are formatted HPFS (OS/2, and an OS/2 maintenance
partition,
 MR> perhaps), then I'll be finding several candidate partitions.
 MR>
 MR> Is there a simple way to figure out which one is correct?  Perhaps
 MR> there's a value for the partition status indicating whether the
 MR> partition is accessible or not?

 JDBP> Partitions are "hidden" by programs such as Boot Manager by
changing
 JDBP> their types to an "unrecognised" type.  Usually this is done by
just
 JDBP> flipping a bit or two in the type byte.  So the answer is that if
you
 JDBP> have "hidden" partitions they will usually have types that, as
 JDBP> standard, operating systems will not recognise as valid partition
 JDBP> types.  There will be only one primary partition in the primary
MBR
 JDBP> that has a valid partition type (and that isn't an extended
partition),
 JDBP> and that will be your candidate partition.
 JDBP> The reason that it works this way is because of the way that most
 JDBP> operating systems look for partitions when assigning drive
letters to
 JDBP> them.  They use a very simplistic algorithm.  They stop
processing the
 JDBP> table in each MBR when they reach the first (non-0x05/0x0F)
partition
 JDBP> with a type that they recognise. ( If you want to see how OS/2
does
 JDBP> it, look at the code for OS2DASD.DMD on the OS/2 DDK, and look at
the
 JDBP> Process_Partition function in the file DMBPB.C . )
 JDBP> The slight exception to this rule is Windows NT 4.0, which caters
for
 JDBP> more than one primary partition having a recognised type in the
primary
 JDBP> MBR.  This allows for up to four primary partitions per disk.
But as
 JDBP> the prominent warning displayed by Windows NT's Disk Manager
utility
 JDBP> when one tries to create more than one primary partition
explains, this
 JDBP> is incompatible with other operating systems.  (Well, *most*
other
 JDBP> operating systems.  (-:)

I wish Fido were faster :)

I had since already come to that conclusion, by managing to create two
primary partitions on another machine, formatted HPFS.  OS/2 sets the
type
to 0x17 when hidden, so the one and only 0x07 in the partition table
must
be the correct one, whether or not the volume in question is a primary
partition or a logical drive in the extended partition.

 JDBP> I thought that you had already solved this problem, though.
Didn't
 JDBP> you say that the DosDevIOCtl to obtain the BPB gave you in the
 JDBP> "reserved sectors" field the offset from the start of the "drive"
to
 JDBP> the partition ?  Surely that solves this very problem ?

Actually, I was referring to the "hidden sectors" field.  As I
implemented
it, it did not work.  I was just adding that value to the sector number
of
the TrackTable[] entry.  If I used that value to get a proper CHS value,
it
might work.

As it is, I've already done the partition table work.  I read in the
first
sector of the "logical drive", and if it ends with 0x55AA, I consider
bytes
446 through 509 to be the partition table.  From there, the first (and
necessarily only) type 0x07 partition found is what's desired.  Since
the
CHS values aren't necessarily useful (consider a logical drive that's
entirely beyond the 1024th cylinder), I just use the boot sector
location.
Add that to the LSN, convert to a CHS value, and DosDevIOCtl() gets what
it's supposed to.  It works on every type of partition I have (I tested
finding FAT drives, too).

I've put all the functions I've been playing around with into a single
program now, which will set the HPFS dirty bit, report the free space
discrepancy, report the total system usage, and dump any sector on the
drive.  Once I think of a few more pseudo-useful things to do, I'll put
it
up for download.

Then I might try playing around with a HPFS undelete program, which
brings
me to a question about something you mentioned previously.

It does stand to reason that any FNode which resides in a sector marked
free is a deleted file, that could potentially be recovered.  What about
FNodes that happen to reside in the reserved but as-yet unused sectors
of a
file?  That is, when a file is created with some slack space to grow
into,
it's slack space will be marked used, but not written to yet.  Could
there
not be an FNode, or even file data, in that slack space which can be
recovered?

Mike Ruskai
thannymeister@yahoo.com

... Arguing logic with a programmer can get you hexed.

___ Blue Wave/QWK v2.20





--- Platinum Xpress/Win/Wildcat5! v3.0pr2
 * Origin: FIDO QWK MAIL & MORE!  WWW.DOCSPLACE.ORG (1:3603/140)

+----------------------------------------------------------------------------+

From: MIKE RUSKAI                                       09-Nov-99 17:37:08
  To: JONATHAN DE BOYNE POLLARD                         09-Nov-99 22:40:27
Subj: PROMPT $I in PMCMD

[This may be a duplicate message.  My Fido host had a glitch]
[that prevented mail from going out for an unspecified      ]
[length of time, so I have to repost recent messages.       ]


Some senseless babbling from Jonathan De Boyne Pollard to George White
on 11-03-99  09:49 about PROMPT $I in PMCMD...

 JdBP>> Actually, what I need to do is to write a control window that
 JdBP>> can be used to display coloured text, rather than relying on a
 JdBP>> standard MLE control.  I know that these things can be done,
 JdBP>> because programs like ZOC and LiveWire use such controls for
 JdBP>> terminal emulator windows.  Does anyone know how they do it ?
 JdBP>> I've tried rolling my own, but it is *very* slow in repainting
 JdBP>> (all of those calls to GpiCharStringPosAt() take a long time).

 GW> I seem to remember that a way to do this was to hide the window,
 GW> build the text, and then make it visible.

 JDBP> That isn't the solution.  The text is already "built", in an
internal
 JDBP> logical screen buffer.  The problem is that the raw GPI call to
 JDBP> actually paint the character cells onto the device context is
slow, and
 JDBP> that often many characters, in different combinations of colours,
have
 JDBP> to be redrawn during a WM_PAINT event when the window contents
are
 JDBP> being redisplayed.
 JDBP> Does anyone here in OS2PROG have any suggestions ?

I can't be sure, because I don't have either program installed, but I
thought both ZOC and LiveWire actually used a VIO window for the
terminal.

That would mean a bunch of calls to VioSetState() to set the color
registers.  It'd probably be more work, but if that's what ZOC and
LiveWire
used, it must be faster.

Unless, of course, you rely on ANSI for colors.  In that case, you'd
just
need to put the control codes into the text you write.

I think I recall a comment from the author of either ZOC or LiveWire,
years
ago, explaining why flashing ANSI characters instead have a grey
background, because of how VIO ANSI is handled.

Mike Ruskai
thannymeister@yahoo.com

... BBS addiction is a Terminal disease.

___ Blue Wave/QWK v2.20

--- Platinum Xpress/Win/Wildcat5! v3.0pr2
 * Origin: FIDO QWK MAIL & MORE!  WWW.DOCSPLACE.ORG (1:3603/140)

+----------------------------------------------------------------------------+

From: Jonathan de Boyne Pollard                         08-Nov-99 09:40:23
  To: MIKE RUSKAI                                       10-Nov-99 07:24:21
Subj: Multiple primary HPFS

 MR> It does stand to reason that any FNode which resides in a sector
 MR> marked free is a deleted file, that could potentially be recovered. 
 MR> What about FNodes that happen to reside in the reserved but as-yet
 MR> unused sectors of a file?  That is, when a file is created with some
 MR> slack space to grow into, it's slack space will be marked used, but
 MR> not written to yet.  Could there not be an FNode, or even file data,
 MR> in that slack space which can be recovered?

This situation only ocurrs when a file is in an intermediate state.  The slack 
claimed space beyond the end of the file will be released when (the last open
handle to) the file is closed.  If this were not done, the on-disc state would 
be inconsistent.  (Also, programs that guessed wrong as to the sizes of files
that they need would cause a great deal of space to be wasted unnecessarily.)  
CHKDSK treats the case where the allocated space exceeds the file length given 
in the FNode as an error, and corrects it.

If you are looking directly at the disc data, then it follows that no open
file handles exist for files on the volume that you are writing to, since you
will have locked the volume.  It thus follows that no file should be in that
particular intermediate state.  If a file *is* in such a state, then be aware
that there was probably a dirty shutdown and that the volume data may thus be
inconsistent elsewhere as well.  Tread carefully.  You don't really want to
write *any* new data (such as new directory entries for files that you have
undeleted, for example) to that volume before running CHKDSK to make the
on-disc structures self-consistent again.

  JdeBP 

--- FleetStreet 1.22 NR
 * Origin: JdeBP's point, using Squish <yuk!> (2:257/609.3)

+----------------------------------------------------------------------------+

From: Jonathan de Boyne Pollard                         08-Nov-99 09:51:11
  To: MIKE RUSKAI                                       10-Nov-99 07:24:21
Subj: PROMPT $I in PMCMD

 JdBP>>> Actually, what I need to do is to write a control window that
 JdBP>>> can be used to display coloured text, rather than relying on a
 JdBP>>> standard MLE control.  I know that these things can be done,
 JdBP>>> because programs like ZOC and LiveWire use such controls for
 JdBP>>> terminal emulator windows.  Does anyone know how they do it ?
 JdBP>>> I've tried rolling my own, but it is *very* slow in repainting
 JdBP>>> (all of those calls to GpiCharStringPosAt() take a long time).

 MR> I can't be sure, because I don't have either program installed, but I
 MR> thought both ZOC and LiveWire actually used a VIO window for the
 MR> terminal.
 MR>
 MR> That would mean a bunch of calls to VioSetState() to set the color
 MR> registers.  It'd probably be more work, but if that's what ZOC and
 MR> LiveWire used, it must be faster.

In the case of PM, a VIO presentation space is simply a handy way of not
having to do the raw character painting onesself.  Unfortunately, in a pure
32-bit application the option of a VIO presentation space is not open to me.

So I'm looking for an efficient way of drawing characters to a presentation
space during a WM_PAINT event myself.  I've constructed a logical screen
buffer, containing the character and attribute pairs, and I want to repaint
the presentation space as appropriate whenever a WM_PAINT comes along.  I used 
GpiCharStringPosAt(), but it is very slow when drawing a single character at a 
time.  Unfortunately, I have to draw single characters at a time when it is
the case that successive characters have different foreground and background
colours.

There must *be* a way to do this, because the VIO library seems to draw
characters (taken from its logical character/attribute video buffer) onto a
presentation space very rapidly.  I was hoping that someone would know what it 
is.

  JdeBP 

--- FleetStreet 1.22 NR
 * Origin: JdeBP's point, using Squish <yuk!> (2:257/609.3)

+----------------------------------------------------------------------------+

From: Jonathan de Boyne Pollard                         08-Nov-99 10:01:02
  To: MIKE RUSKAI                                       10-Nov-99 07:24:21
Subj: Multiple primary HPFS

 MR> I wish Fido were faster :)

It's not necessarily Fidonet.  I sometimes choose to work on things like TAU
instead of reading messages, and then I go through the backlog of messages
that have accumulated in my messagebase over several days.

  JdeBP 

--- FleetStreet 1.22 NR
 * Origin: JdeBP's point, using Squish <yuk!> (2:257/609.3)

+----------------------------------------------------------------------------+

From: MIKE RUSKAI                                       10-Nov-99 19:51:00
  To: JONATHAN DE BOYNE POLLARD                         11-Nov-99 11:15:15
Subj: Multiple primary HPFS

Some senseless babbling from Jonathan De Boyne Pollard to Mike Ruskai
on 11-08-99  09:40 about Multiple primary HPFS...

 MR> It does stand to reason that any FNode which resides in a sector
 MR> marked free is a deleted file, that could potentially be recovered. 
 MR> What about FNodes that happen to reside in the reserved but as-yet
 MR> unused sectors of a file?  That is, when a file is created with some
 MR> slack space to grow into, it's slack space will be marked used, but
 MR> not written to yet.  Could there not be an FNode, or even file data,
 MR> in that slack space which can be recovered?

 JDBP> This situation only ocurrs when a file is in an intermediate state. 
 JDBP> The slack claimed space beyond the end of the file will be released
 JDBP> when (the last open handle to) the file is closed.  If this were not
 JDBP> done, the on-disc state would be inconsistent.  (Also, programs that
 JDBP> guessed wrong as to the sizes of files that they need would cause a
 JDBP> great deal of space to be wasted unnecessarily.)  CHKDSK treats the
 JDBP> case where the allocated space exceeds the file length given in the
 JDBP> FNode as an error, and corrects it. 

Sure, but as most people run an undelete program shortly after accidentally
deleting that which is not backed up, and don't close anything that might
write to the disk, wouldn't it be worth checking for such situations in an
undelete program?

That is, is there any reason not to look at all space, regardless of
whether or not it's marked as in use?

 JDBP> If you are looking directly at the disc data, then it follows that no
 JDBP> open file handles exist for files on the volume that you are writing
 JDBP> to, since you will have locked the volume.  It thus follows that no
 JDBP> file should be in that particular intermediate state.  If a file *is*
 JDBP> in such a state, then be aware that there was probably a dirty shutdown
 JDBP> and that the volume data may thus be inconsistent elsewhere as well. 
 JDBP> Tread carefully.  You don't really want to write *any* new data (such
 JDBP> as new directory entries for files that you have undeleted, for
 JDBP> example) to that volume before running CHKDSK to make the on-disc
 JDBP> structures self-consistent again. 

I would take the same route as GammaTech's undelete program, by not writing
anything at all to the drive in question, but merely reading from it, and
writing undeleted files to a separate drive.

That would make locking the drive unnecessary, which also makes it possible
to function on a drive that can't be locked (such as the boot drive).

Mike Ruskai
thannymeister@yahoo.com


... A cat will always sit on whatever it is you're trying to read or copy.

___ Blue Wave/QWK v2.20
--- Platinum Xpress/Win/Wildcat5! v3.0pr2
 * Origin: FIDO QWK MAIL & MORE!  WWW.DOCSPLACE.ORG (1:3603/140)

+----------------------------------------------------------------------------+

From: Coridon Henshaw                                   11-Nov-99 22:24:05
  To: Jonathan de Boyne Pollard                         12-Nov-99 10:11:00
Subj: PROMPT $I in PMCMD

On Monday November 08 1999 at 09:51, Jonathan de Boyne Pollard wrote to MIKE
RUSKAI:

 JP> I used GpiCharStringPosAt(), but it is very slow when drawing a
 JP> single character at a time.  Unfortunately, I have to draw single
 JP> characters at a time when it is the case that successive characters
 JP> have different foreground and background colours.

I know this is outside your critera, but have you tried rendering your virtual 
buffer into a bitmap (using your own font rasterization routines) rather than
relying on GPI and a presentation space?

--- GoldED/2 3.0.1
 * Origin: Life sucks and then you croak. (1:250/820)

+----------------------------------------------------------------------------+

+============================================================================+
