
                    Programming/using OS/2 REXX      (echo)

                  Sunday, 26-Sep-1999 to Friday, 01-Oct-1999

+----------------------------------------------------------------------------+

From: Peter Knapper                                     26-Sep-99 11:48:28
  To: Mike Ruskai                                       26-Sep-99 09:33:16
Subj: hstart wget?

Hi Mike,

 MR> I'm currently working on a design that will allow up to 
 MR> eight simultaneous transfers (using the threading support 
 MR> of OREXX).

This sentence caught my eye, I have not dabbled in OREXX too much (actually I
have tried to avoid anything to do with OBJECTS until I absolutely HAVE
to......;-)), but can you please elaborate on how OREXX implements
Multi-threading? Is each thread under explicit coder control (IE can a control 
thread stop/start other threads?), or is it "dynamic" threading within OREXX
that does not allow explicit thread control? 

I have implemented multi-threading in C many years ago, and have a project for 
Rexx that could use it nicely, but I may have to dig more into OREXX to get to 
grips with the requirements for using it.

Cheers..........pk.


--- Maximus/2 3.01
 * Origin: Another Good Point About OS/2 (3:772/1.10)
30

+----------------------------------------------------------------------------+

From: John Clarke                                       25-Sep-99 09:02:10
  To: Eddy Thilleman                                    26-Sep-99 14:55:17
Subj: hstart wget?

On Sep 21, 1999 at 23:41 hrs, Eddy Thilleman of 2:500/143.7 wrote to All:

Hello Eddy,

 ET> I made a REXX file wgetmir.cmd to read all the lines in a textfile 
 ET> which are the websites or files on internet to retrieve with wget. 
 ET> wget is running OK when hstarted from a command line or a plain batch 
 ET> file, but not from the REXX file (no screen output or no log file if 
 ET> log file is specified). Does someone spot what I'm doing wrong?

I can't comment of the rexx file but I can tell you how I have wget running
here.

I have a series of icons in a folder.  Each icon calls wget and points to a
different directory on the disk - ie one for each of DOS, Games, OS/2, Windows 
etc.  I put a file in these directories called filelist.lst that contains the
list of URLs.  The icons have the following command lines

Path and filename
c:\wget\wget.exe

optional parameters
-v -id:\down-dos\filelist.lst -P d:\down-dos
or
-v -id:\down-win\filelist.lst -P d:\down-win
etc

Working directory
c:\wget

This puts the files in d:\down-dos or d:\down-win etc and writes the output to 
screen.  I haven't found a way to put it to screen and to a logfile at the
same time - it seems to be one or the other.

All I do if put the file filelist.lst in the appropriate directory and then
double click on the icon.  I've had this running for over a year now.

The only improvement I would like would be the ability queue the 'icons' if I
have several filelist.lst URL files.  Something along the lines of

if exist d:\down-dos\fileslist.lst then
       call get_dos
if exist d:\down-win\filelist.lst then
       call get_win
etc etc.

If you get your rexx script working would you please send me a copy?  I'll see 
if I can add the queue processing.

I hope this helps.

Regards ... John

Email: jclarke@zip.com.au

--- MsgedSQ/2 3.30
 * Origin: The Clearing House (3:713/730)
267/200
45

+----------------------------------------------------------------------------+

From: Sergey Popov                                      26-Sep-99 12:40:11
  To: Peter Knapper                                     26-Sep-99 23:20:00
Subj: Re: Concurrent sessions fail...

ਢ , Peter!

25  99 13:31, Peter Knapper -> All:

 PK> Hi Folks,

 PK> I have an odd problem here folks, I have a REXX.CMD file that I invoke in
 PK> 2 different sessions takinging 2 different sets of parameters, and while
 PK> this works fine 90% of the time, sometimes one of the invoked sessions
 PK> fails to start up and terminates early with NO output. Usually it is the
 PK> FIRST session STARTed that fails to activate. Here is the flow of control
 PK> for the entire process -

It could help you but I've got the similar odd problem with my CREXX code
while
not recognize.... my HDD is too slow! I did the same as you - insert "Call
SysSleep nn" for wating while my HDD stoping :-(

With best regards, Sergey Popov.

--- GoldED/W32 3.0.1
 * Origin:  ! ଠ ਠ ! (2:5085/1.104)
140/1

+----------------------------------------------------------------------------+

From: Eddy Thilleman                                    25-Sep-99 10:43:12
  To: Jack Stein                                        27-Sep-99 04:07:05
Subj: Classic REXX files?

Hello Jack,

22 Sep 99 08:34, Jack Stein wrote to Eddy Thilleman:

ET>> Then you start your startup.cmd file with a REXX comment?

JS> Yes, one can't call REXX functions from a batch file, right?

That's for sure. :)

ET>> That's what I do: I call a REXX file from my startup.cmd.

JS> That's OK too, but not necessary if your startup.cmd is a REXX file.

I know.

JS> Yep, but you must understand how to use regular batch commands in a
JS> REXX script if you are also mixing in regular batch commands.

Quote them. :)

  Greetings   -=Eddy=-        email: eddy.thilleman@net.hcc.nl

... Confucius say: Let them eat Twinkies!
--- GoldED/2 3.0.1
 * Origin: Windows95 is a graphic DOS extender (2:500/143.7)

+----------------------------------------------------------------------------+

From: Steve McCrystal                                   26-Sep-99 08:27:10
  To: Eddy Thilleman                                    27-Sep-99 19:40:01
Subj: Classic REXX files?

;
In a msg of <Wednesday September 22 1999>, Eddy Thilleman writes to Steve
Mccrystal:
;
Eddy,

 ET> If you add REXX code to a plain batch file then that batch is not
 ET> a REXX file that can be run, you have to turn the whole batch
 ET> file into a REXX file: on the first line in the first column the
 ET> batch file must start with a comment, to begin with.

 As I mentioned earlier, I knew I must put a comment in as the first line of
STARTUP.CMD in order for it to be a REXX file.  I just forgot to do it! :(

Done correctly, it *does* work just fine!

My thanks to you and the others who reminded me!

-[Steve]-

--- GoldED/2 3.0.1/#
 * Origin: -[Steve's Place]- New Berlin, WI (FidoNet 1:154/731.2)

+----------------------------------------------------------------------------+

From: Eddy Thilleman                                    26-Sep-99 12:04:10
  To: Mike Ruskai                                       27-Sep-99 19:40:01
Subj: hstart wget?

Hello Mike,

24 Sep 99 13:15, MIKE RUSKAI wrote to EDDY THILLEMAN:

ET>> LineIn( InFile, 1, 0 )
ET>> /* No read; opens inputfile */
ET>> /* (if file is already open, sets the read position to the first

MR> You shouldn't depend on this.

Please explain why.

MR> Opening files should always be done with the stream() function (or
MR> with a Stream object in OREXX), like this:

Please explain why files should be opened with the stream() function?

MR> http://hobbes.nmsu.edu/pub/os2/apps/internet/suite/rxiutl??.zip

Maybe I might do take a look at it.

MR> and has a better transfer status display than WGET.

With wget, I want to use and a log file and a have status display on screen
(not possible now AFAICS).

MR> My goal is to support all of the functionality of WGET.

That's a lot of work.

  Greetings   -=Eddy=-        email: eddy.thilleman@net.hcc.nl

... Windows Error 003: Operator died during wait.
--- GoldED/2 3.0.1
 * Origin: Windows98 is a graphic DOS extender (2:500/143.7)

+----------------------------------------------------------------------------+

From: Eddy Thilleman                                    26-Sep-99 21:02:06
  To: Peter Knapper                                     27-Sep-99 19:40:01
Subj: Concurrent sessions fail...

Hello Peter,

25 Sep 99 13:31, Peter Knapper wrote to All:

PK> I have an odd problem here folks, I have a REXX.CMD file that I invoke
PK> in 2 different sessions takinging 2 different sets of parameters, and
PK> while this works fine 90% of the time, sometimes one of the invoked
PK> sessions fails to start up and terminates early with NO output.

I might have a similar problem here, I see the same symptoms.

PK> Usually it is the FIRST session STARTed that fails to activate.

Here it's only the last session STARTed that executes, the previous ones seem
to have not executed.

I have one or more text files with files/websites to retrieve from internet
and I run a REXX script that reads the lines from that text file and for each
line it STARTs (with HSTART) a separate session with a wget to retrieve it (no 
REXX script in each session, just a command to wget).

To see if a short pause will fix it, I try it next time with "call SysSleep 1" 
following the START of each session.

Here is my REXX file:

------------------- begin of wgetmir.cmd -------------------
/* start wget mirroring each website listed in wget.mir */
arg InFile

if InFile = '' then
  InFile = 'wget.mir'

dir = directory( 'c:\www' )
'@set wgetrc=c:\www\wget.cfg'

cmd1 = 'wget.exe -m -c -np --tries=1 -k '
cmd2 = ' -a '
LogFile = "wget.mir.log"
filler = copies('-',30)

signal on notready

LineNr = 0

LineIn( InFile, 1, 0 )
  /* No read; opens inputfile */
  /* (if file is already open, sets the read position to the first line). */

do while Lines( InFile ) > 0
  LineNr = LineNr + 1
  Line = LineIn( InFile )
  say LineNr': 'Line
  LogFile = "wget.mir"LineNr".log"
  cmd = cmd1 || Line || cmd2 || LogFile
  written = LineOut( LogFile, filler date('W') date() Time() filler )
  "@start /win /c /b "cmd">nul"
  call SysSleep 1
end

exit

NotReady:
  if LineNr = 0 then
    say 'could not open 'InFile

exit
-------------------- end of wgetmir.cmd --------------------

.....later.....

I wondered why wget's output doesn't show up in the logfiles, so I started the 
sessions with /k instead of /c to see more, then I see errormessage:
"wet.mir1.log: Permission denied".

It turned out that the call to SysSleep apparently prevents wget from opening
the logfile.

I don't know why?

So I removed the call to SysSleep and (ofcourse changed /k back to /c) and now 
it works as it should.

PK> Does anyone have any idea of how I can remove the SLEEP and keep the
PK> 2 sessions reliably opening?

I don't know the answer.

  Greetings   -=Eddy=-        email: eddy.thilleman@net.hcc.nl

... C:\WINDOWS    C:\WINDOWS\GO    C:\PC\CRAWL
--- GoldED/2 3.0.1
 * Origin: Windows98 is a graphic DOS extender (2:500/143.7)

+----------------------------------------------------------------------------+

From: MIKE RUSKAI                                       26-Sep-99 23:17:00
  To: PETER KNAPPER                                     27-Sep-99 19:40:01
Subj: hstart wget?

Some senseless babbling from Peter Knapper to Mike Ruskai
on 09-26-99  11:48 about hstart wget?...

 PK> Hi Mike,
 
 MR> I'm currently working on a design that will allow up to 
 MR> eight simultaneous transfers (using the threading support 
 MR> of OREXX).

 PK> This sentence caught my eye, I have not dabbled in OREXX too much
 PK> (actually I have tried to avoid anything to do with OBJECTS until I
 PK> absolutely HAVE to......;-)), but can you please elaborate on how OREXX
 PK> implements Multi-threading? Is each thread under explicit coder control
 PK> (IE can a control thread stop/start other threads?), or is it "dynamic"
 PK> threading within OREXX that does not allow explicit thread control? 

It's nothing as direct as using the OS/2 API.  The way to run a separate 
thread is to issue an early reply from an object method (and perhaps also a 
routine), rather than returning, allowing the method to continue running on 
a separate thread.  The reply may pass data, but the subsequent return that 
ends the method can return no data (for obvious reasons).

I've already used this for monitoring the transfer with the urlget.cmd at 
Hobbes.  I'm not sure if there's a better way, but I've taken to using 
queues to exchange data between different threads, since there seems to 
never be any conflict when queueing and pulling from the same queue at the 
same time.

What I'll end up doing is using a separate thread to write all information 
to the screen, which will "poll" all the information queues at a set 
interval (i.e. check for items in the queues, and if present, retrieve them 
for action).

 PK> I have implemented multi-threading in C many years ago, and have a
 PK> project for Rexx that could use it nicely, but I may have to dig more
 PK> into OREXX to get to grips with the requirements for using it.

You'll have to modify it considerably, because the threading is not quite 
direct, as I said.

But it provides for some interesting possibilities.  Here's a brief example 
to demonstrate the concept:

/* Simple threading example */
call RxFuncAdd 'SysLoadFuncs','RexxUtil','SysLoadFuncs'
call SysLoadFuncs

.local['DQ']=.queue~new
.local['CON']=.stream~new('stdout')

scrupd=.ScreenUpdater~new
scrupd~StartUpdating

do i=1 to 1000
    .dq~queue(i)
end
.dq~queue('EXIT')

do until .dq~items=0
    call SysSleep 0.10
end

exit

::class ScreenUpdater

::method init
expose row col
call SysCls
call SysCurState 'off'
call SysCurPos 11,0
.con~~charout('Data received: ')~flush
parse value SysCurPos() with row col

::method StartUpdating
expose row col
reply
done=.false
do until done
    if .dq~items>0 then do
        data=.dq~pull
        if data='EXIT' then done=.true
        else do
            call SysCurPos row,col
            .con~~charout(data~right(5))~flush
        end
    end
end
say ''
return
/* end example */

As might be apparent, there's no way to kill the thread from without.  It's 
important to establish a signal for termination.  In the above case, it's 
merely stuffing the queue with 'EXIT', then waiting until the queue is 
empty before exiting.  If you remove that wait, you'll notice that you're 
back at the command prompt while the thread is still running and updating 
the screen (it is, after all, a thread of the current command processor's 
process).

Mike Ruskai
thannymeister@yahoo.com


... Cannibals don't eat lawyers. Professional courtesy.

___ Blue Wave/QWK v2.20
--- Platinum Xpress/Win/Wildcat5! v3.0pr2
 * Origin: FIDO QWK MAIL & MORE!  WWW.DOCSPLACE.ORG (1:3603/140)
267/200
45

+----------------------------------------------------------------------------+

From: David Noon                                        26-Sep-99 20:23:00
  To: Steve McCrystal                                   28-Sep-99 04:45:27
Subj: Classic REXX files?

In a message dated 09-22-99, Jack Stein said to Steve Mccrystal about
"Classic REXX files?"

Hi Steve,

JS>For example, to load my 4OS2 alias file a regular cmd file would have:

JS>ALIAS /r e:\4os2301\os2alias

JS>REXX has no clue what "ALIAS" is, so he passes ALIAS to CMD.EXE.  The 
JS>paramaters however are very confusing to REXX, since it 
JS>recognizes some of the symbols, such as "/", and "\".  
JS>Because of this, you must quote the parameters so REXX 
JS>doesn't try to divide something by r, for example, so the 
JS>line looks like this in a REXX script:

JS>ALIAS '/r e:\4os2301\os2alias'

A more idiomatic approach would be to code it as:

   "ALIAS /r e:\4os2301\os2alias"

or better still:

   ADDRESS 'CMD' "ALIAS /r e:\4os2301\os2alias"

JS>Also, to make a directory, the same thing applies:

JS>md "z:\xpress"

There is a perfectly good REXXUTIL function called SysMkDir that will do
this without requiring "shelling out" to run MD. The preferred way is:

     CALL SysMkDir "z:\xpress"

However, this is predicated on you having registered the REXXUTIL functions
before calling SysMkDir. And as Jack rightly says ...

JS> In short, it's OK not to start up the extended utilities 
JS>in your startup.cmd, but it's not OK to not know why you 
JS>got the errors.:-)

But to continue on theme of not "shelling out", you should avoid assumptions
whenever you are writing code. Invoking an external command through a shell
nearly always involves _assuming_ that your current subcommand handler is
CMD.EXE (or some moral equivalent). This assumption is not always valid!

[Sidebar: When the REXX interpreter encounters some "statement" it cannot
comprehend, that statement is treated as a subcommand and its text is passed
to the default subcommand handler. This is what is happening with Jack's
ALIAS statement, above.]

Some of us like to use REXX to customize terminal programs (e.g. ZOC), text
editors (e.g. EPM, LPEX) and other programs. In such a context, the program
being customized will often become the default subcommand handler. Thus,
should you code an OS/2 command in your REXX program it will be passed to
whatever program established itself as the subcommand handler, not directly
to CMD.EXE (or whatever you have in OS2_SHELL). The subcommand handler is
then responsible for passing it along to CMD.EXE. This often involves
creating a separate address space in which to run CMD.EXE.

You can pass your commands directly to CMD.EXE if you bypass the default and
use an explicit ADDRESS statement, as I did above with Jack's ALIAS command.
If CMD.EXE is not your shell then this will still create a separate address
space in which to run the command, but it will avoid having to go through
some intermediary.

Creating a separate address space is generally an undesirable arrangement.
Firstly, it is slow. Secondly, it alters the semantics of many commands.
There are situations where it is unavoidable, though. Specifically, any
command that is actually a external program must be run in a separate
address space anyway. However, it is the semantic differences that usually
bite the REXX programmer.

The classic example is using the CD (or CHDIR) command to change the current
working directory. A lot of newbie (and some experienced) REXX programmers
code:

    "CD \dir\subdir"

to change the directory and then find that, apparently, nothing happened.
What actually happened was that a new address space was created to run
CMD.EXE, the working directory for that new address space was changed, and
then the address space terminated, returning control to the REXX program.

Instead, the programmer should have coded:

     CALL DIRECTORY "\dir\subdir"

to use the built-in DIRECTORY function. This is always executed within the
current address space, so its results "stick".

Similarly, one should use the VALUE built-in function to query and set
environment variables. This function always runs within the current address
space and use that address space's environment block.

The general moral to my story is that one should use REXX functions calls
whenever and wherever they are available, as the results are predictable.
Shelling out to run external commands often produces very unexpected
results. The upshot is that one should always check to see if REXX or
REXXUTIL offers a built-in feature before shelling out to run a command.

Regards

Dave
<Team PL/I>
___
 * MR/2 2.25 #353 * "Who is #1?"  "You are, #6."

--- Maximus/2 3.01
 * Origin: Air Applewood, OS/2 Gateway to Essex 44-1279-792300 (2:257/609)
267/200
45

+----------------------------------------------------------------------------+

From: Peter Knapper                                     28-Sep-99 22:57:20
  To: Mike Ruskai                                       28-Sep-99 11:30:02
Subj: hstart wget?

Hi Mike,


 PK> can you please elaborate on how OREXX implements 
 PK> Multi-threading? Is each thread under explicit coder 
 PK> control (IE can a control thread stop/start other threads?), 
 PK> or is it "dynamic"threading within OREXX that does not 
 PK> allow explicit thread control? 

 MR> It's nothing as direct as using the OS/2 API.  The way to run a 
 MR> separate thread is to issue an early reply from an object method 
 MR> (and perhaps also a routine), rather than returning, allowing 
 MR> the method to continue running on a separate thread.  

Hmmmm, thanks for the information, it was very useful. I will have to think
about it some more before I decide to head down that path. 

Cheers............pk.


--- Maximus/2 3.01
 * Origin: Another Good Point About OS/2 (3:772/1.10)
30

+----------------------------------------------------------------------------+

From: Eddy Thilleman                                    27-Sep-99 14:00:13
  To: John Clarke                                       29-Sep-99 05:46:22
Subj: hstart wget?

Hello John,

25 Sep 99 09:02, John Clarke wrote to Eddy Thilleman:

JC> I have a series of icons in a folder.  Each icon calls wget and points
JC> to a different directory on the disk - ie one for each of DOS, Games,
JC> OS/2, Windows etc.  I put a file in these directories called
JC> filelist.lst that contains the list of URLs.

On connect, Injoy autostarts a batch file which starts (among other things)
several separate sessions with wget (called with different options, for
retrieving files from internet, for mirroring (parts of) websites - 'quotum' 1 
MB per session at a time).

JC> This puts the files in d:\down-dos or d:\down-win etc and writes the
JC> output to screen.  I haven't found a way to put it to screen and to a
JC> logfile at the same time - it seems to be one or the other.

I came to the same conclusion. I am thinking about putting a message in the
wget mailing list to ask output to the screen and to a logfile at the same
time.

JC> All I do if put the file filelist.lst in the appropriate directory
JC> and then double click on the icon.  I've had this running for over a
JC> year now.

I don't doubleclick, I have it on automatic. :)

JC> If you get your rexx script working would you please send me a copy?
JC> I'll see if I can add the queue processing.

I sent yesterday my REXX file again, have you seen it?

  Greetings   -=Eddy=-        email: eddy.thilleman@net.hcc.nl

... C:\DOS\RUN   C:\WINDOWS\CRAWL   C:\OS2\FLY !
--- GoldED/2 3.0.1
 * Origin: Windows95 is a graphic DOS extender (2:500/143.7)

+----------------------------------------------------------------------------+

From: Eddy Thilleman                                    27-Sep-99 21:45:19
  To: Mike Ruskai                                       29-Sep-99 05:46:22
Subj: hstart wget?

Hello Mike,

24 Sep 99 13:15, MIKE RUSKAI wrote to EDDY THILLEMAN:

MR> You shouldn't depend on this.  Opening files should always be done
MR> with the stream() function (or with a Stream object in OREXX),

I would like to know why.

MR> like this:
MR> check=stream(InFile,'c','open read') if check='' then /* couldn't
MR> open the file, act appropriately */

So I changed my REXX file into this:

------------------- begin of wgetmir.cmd -------------------
/* start wget mirroring each website/file listed in wget.mir */
arg InFile

if InFile = '' then
  InFile = 'wget.mir'

dir = directory( 'c:\www' )
'@set wgetrc=c:\www\wget.cfg'

cmd1 = 'wget.exe -m -c -np --tries=1 -k '
cmd2 = ' -a '
LogFile = "wget.mir.log"
filler = copies('-',30)

signal on notready

LineNr = 0

check = stream( InFile, 'c', 'open read' )
if check != 'READY' then  /* couldn't open the file, act appropriately */
  do
    say "Cannot open file" InFile check
    ch = SysGetKey( NoEcho )
  end
else
  do while stream( InFile, 'state' ) = 'READY'
    LineNr = LineNr + 1
    Line = LineIn( InFile )
    LogFile = "wget.mir"LineNr".log"
    say LineNr': 'Line
    cmd = cmd1 || Line || cmd2 || LogFile
    written = LineOut( LogFile, filler date('W') date() Time() filler )
    "@start /win /c /b "cmd
  end

check = stream( InFile, 'c', 'close' )

return

NotReady:
  if LineNr = 0 then
    say 'NotReady' InFile

return
-------------------- end of wgetmir.cmd --------------------

  Greetings   -=Eddy=-        email: eddy.thilleman@net.hcc.nl

... WindowError:00E Window open, do not look in.
--- GoldED/2 3.0.1
 * Origin: Windows95 is a graphic DOS extender (2:500/143.7)

+----------------------------------------------------------------------------+

From: David Noon                                        27-Sep-99 18:34:00
  To: Peter Knapper                                     29-Sep-99 05:46:22
Subj: Multi-threaded OREXX

In a message dated 09-26-99, Peter Knapper said to Mike Ruskai about "hstart
wget?"

Hi Peter,

 MR> eight simultaneous transfers (using the threading support 
 MR> of OREXX).

PK>This sentence caught my eye, I have not dabbled in OREXX too much
PK>(actually I have tried to avoid anything to do with OBJECTS until I 
PK>absolutely HAVE to......;-)),

If you want to use native multi-threading in OREXX you HAVE to.

PK>but can you please elaborate 
PK>on how OREXX implements Multi-threading? Is each thread 
PK>under explicit coder control (IE can a control thread 
PK>stop/start other threads?), or is it "dynamic" threading 
PK>within OREXX that does not allow explicit thread control? 

It is quite simple really. An object class can have one or more methods that
run in secondary threads. These are identified by the REPLY statement. This
statement sends the nominal result of the method call back to the caller,
and then allows the method to continue. This has the effect of resuming the
caller's execution and allowing the method to continue in parallel. The
obvious implementation is to use OS/2's threading capability.

You get to control thread creation in your code. You can have as many
threads in your address space as virtual storage constraints and the system
thread count constraint permit.

As to synchronizing concurrent threads, that is up to you. The simplest
approach is to add an attribute to the object class and have the
asynchronous method check to see if some other thread has requested that it
terminate. Basically, the other thread sets the attribute to some value and
when the thread inside the owning object checks the attribute it takes the
appropriate action.

A somewhat slicker method is to use the event semaphore support in REXXUTIL.
This is more idiomatic OS/2, as it eliminates polling of attributes. You
simply wait on and post the shared semaphore in whichever threads you want.

Regards

Dave
<Team PL/I>
___
 * MR/2 2.25 #353 * "More hay, Trigger?"  "No thanks, Roy, I'm stuffed!"

--- Maximus/2 3.01
 * Origin: Air Applewood, OS/2 Gateway to Essex 44-1279-792300 (2:257/609)
267/200
45

+----------------------------------------------------------------------------+

From: David Noon                                        27-Sep-99 18:42:01
  To: Peter Knapper                                     29-Sep-99 05:46:22
Subj: Concurrent sessions fail

In a message dated 09-25-99, Peter Knapper said to All about "Concurrent
sessions fail."

Hi Peter,

[snip]
PK>If I modify PRIMARY.CMD and insert a utility BETWEEN the 2 
PK>Start commands that invokes a 2 second "SLEEP", then 
PK>everything appears to start up fine.

The most likely cause of contention in your code with be the RxFuncAdd()
calls and the subsequent library registrations. These all require a mutex
that serializes the shared memory area for the REXX external functions
table. I suspect something in the REXX run-time just shuts down the process
if acquiring the mutex fails.

My best suggestion is to remove the code in question from the worker
routines and ensure that all the libraries required have been registered
before the overall processing starts. One way would be to do all the prep
work in REXX too, and include in this the registration of the external
libraries. This would allow the ftp processes to operate in parallel as you
intend without having to compete for update access to the REXX external
function table.

Regards

Dave
<Team PL/I>
___
 * MR/2 2.25 #353 * A chrysanthemum by any other name would be easier to
spell.

--- Maximus/2 3.01
 * Origin: Air Applewood, OS/2 Gateway to Essex 44-1279-792300 (2:257/609)
267/200
45

+----------------------------------------------------------------------------+

From: Peter Knapper                                     29-Sep-99 20:42:11
  To: David Noon                                        30-Sep-99 08:38:11
Subj: Multi-threaded OREXX

Hi David,

 MR> eight simultaneous transfers (using the threading support 
 MR> of OREXX).

PK>This sentence caught my eye, I have not dabbled in OREXX too much
PK>(actually I have tried to avoid anything to do with OBJECTS until I 
PK>absolutely HAVE to......;-)),

 DN> If you want to use native multi-threading in OREXX you HAVE to.

I feared as much, but then sometimes you need a good push to get moving
again......;-)

PK>but can you please elaborate 
PK>on how OREXX implements Multi-threading? Is each thread 
PK>under explicit coder control (IE can a control thread 
PK>stop/start other threads?), or is it "dynamic" threading 
PK>within OREXX that does not allow explicit thread control? 

 DN> It is quite simple really. An object class can have one 
 DN> or more methods that run in secondary threads. These are 
 DN> identified by the REPLY statement.

  ...< trimmed >...

Thanks for the details, I have filed this with Mikes reply for reference when
I investigate this further.

The actual mention of Threads in Mikes original message just happened to
arrive a couple of hours before I was about to post a message regarding
process control with REXX. 

In one of my projects I am experiencing a situation with multiple independant
REXX tasks that I wish to terminate by external command if a task reaches a
"no progress" stage. Unfortuantely at the determination of this situation
(generally time based), control is with an external executable that is invoked 
from my REXX code, and I was trying to find a TIDY way to issue a "KILL
PID=nnnn" to terminate that task.

Based on Mike initial message I was considering if the REXX Thread
implementation might provide a "better" way to handle this...

More investigation is needed...

Regards..........pk.


--- Maximus/2 3.01
 * Origin: Another Good Point About OS/2 (3:772/1.10)
30

+----------------------------------------------------------------------------+

From: Peter Knapper                                     29-Sep-99 21:09:07
  To: David Noon                                        30-Sep-99 08:38:11
Subj: Concurrent sessions fail

Hi David,

PK>If I modify PRIMARY.CMD and insert a utility BETWEEN the 2 
PK>Start commands that invokes a 2 second "SLEEP", then 
PK>everything appears to start up fine.

 DN> The most likely cause of contention in your code with be the RxFuncAdd()
 DN> calls and the subsequent library registrations. These all require a mutex
 DN> that serializes the shared memory area for the REXX external functions
 DN> table. I suspect something in the REXX run-time just 
 DN> shuts down the process if acquiring the mutex fails.

Thanks for confirming my thoughts on this one........;-)

I call RxFuncAdd for all the libraries I normally use from Startup.Cmd and
dont release them, so the startup code is normally never needed, its just a
"safety net". I have always hard coded the loads using the basis that the load 
attempts to detect them and does not actually load them if already loaded,
however I notice that some people use RxFuncQuery to test for them, and load
only if needed. However your reply suggests that there may actually be an
issue with doing things this way after all... 

Thanks..........pk.


--- Maximus/2 3.01
 * Origin: Another Good Point About OS/2 (3:772/1.10)
30

+----------------------------------------------------------------------------+

From: MIKE RUSKAI                                       28-Sep-99 16:14:00
  To: EDDY THILLEMAN                                    01-Oct-99 05:53:17
Subj: hstart wget?

Some senseless babbling from Eddy Thilleman to Mike Ruskai
on 09-26-99  12:04 about hstart wget?...

 ET> Hello Mike,

 ET> 24 Sep 99 13:15, MIKE RUSKAI wrote to EDDY THILLEMAN:
 
 ET>> LineIn( InFile, 1, 0 )
 ET>> /* No read; opens inputfile */
 ET>> /* (if file is already open, sets the read position to the first
 
 MR> You shouldn't depend on this.

 ET> Please explain why.

Because it's only a side-effect of the current OS/2 implementation of REXX
- it's not by design.

Furthermore, you aren't checking for success in opening the file.

 MR> Opening files should always be done with the stream() function (or
 MR> with a Stream object in OREXX), like this:

 ET> Please explain why files should be opened with the stream() function?

Because that's the designed way of opening files.  If you check the result,
you can determine whether or not the open was successful.
 
 MR> http://hobbes.nmsu.edu/pub/os2/apps/internet/suite/rxiutl??.zip

 ET> Maybe I might do take a look at it.
 
 MR> and has a better transfer status display than WGET.

 ET> With wget, I want to use and a log file and a have status display on
 ET> screen (not possible now AFAICS).

The current version of urlget.cmd will log a transfer and display the
status on screen, though both functions can be disabled.  Here's what the
log entry looks like:

[23 Sep 1999 - 17:12:46] - Successful transfer for:
ftp://service.boulder.ibm.com/software/asd/java/en_us/javainuf.exe
Received 4,822,320 bytes in 1,553.47 seconds (3,104 bytes/second).
Total size is 20,805,936 bytes, resumed from byte 15,983,616.
---------------------------------------------------------------------------
 
 MR> My goal is to support all of the functionality of WGET.

 ET> That's a lot of work.

Not really.  REXX makes things easier.

Mike Ruskai
thannymeister@yahoo.com


... "...we believe that OS/2 is the OS of the 90's" - Bill Gates, Comdex

___ Blue Wave/QWK v2.20
--- Platinum Xpress/Win/Wildcat5! v3.0pr2
 * Origin: FIDO QWK MAIL & MORE!  WWW.DOCSPLACE.ORG (1:3603/140)
267/200
45

+----------------------------------------------------------------------------+

From: Eddy Thilleman                                    28-Sep-99 21:07:22
  To: John Clarke                                       01-Oct-99 05:53:17
Subj: hstart wget?

Hello John,

25 Sep 99 09:02, John Clarke wrote to Eddy Thilleman:

JC> The only improvement I would like would be the ability queue the
JC> 'icons' if I have several filelist.lst URL files.

I put the URL for what I want to get from the internet in a textfile which is
read by my batch- and REXX files which will start wget to get what is listed
in that textfile.

JC> If you get your rexx script working would you please send me a copy?
JC> I'll see if I can add the queue processing.

I've revised my REXX file:

------------------- begin of wgetmir.cmd -------------------
/* start wget mirroring each website/file listed in wget.mir */
arg InFile

if InFile = '' then
  InFile = 'wget.mir'

dir = directory( 'c:\www' )
'@set wgetrc=c:\www\wget.cfg'

cmd1 = 'wget.exe -m -c -np --tries=1 -k '
LogFile = "wget.mir.log"
ErrorFile = "rexx.error"
filler = copies('-',30)

signal on notready

LineNr = 0

check = stream( InFile, 'c', 'open read' )
if check = 'READY:' then
  do while stream( InFile, 'state' ) = 'READY'
    Line = LineIn( InFile )
    if strip( Line ) = '' then  /* if Line not empty */
    do
      LineNr = LineNr + 1
      LogFile = "wget.mir"LineNr".log"
      say LineNr': 'Line
      cmd = cmd1 || Line || ' -a ' || LogFile
      check   = stream( LogFile, 'C', 'open write' )    /* open logfile */
      check   = stream( LogFile, 'C', 'seek <' 0 )      /* seek to end of
logfile */
      written = LineOut( LogFile, filler date('W') date() Time() filler )  /*
append day date time to logfile */
      check   = stream( LogFile, 'C', 'close' )         /* close logfile so
wget can use it */
      "@start /win /c /b" cmd
    end  /* if Line not empty */
  end
else  /* couldn't open the file */
  do  /* --- begin of untested code --- */
    errmsg  = stream( InFile, 'D' )     /* ask description about possible
error */
    check   = stream( ErrorFile, 'C', 'open write' )    /* open errorfile */
    check   = stream( ErrorFile, 'C', 'seek <' 0 )      /* seek to end of
errorfile */
    written = LineOut( ErrorFile, date('W') date() Time() "Cannot open file"
InFile errmsg )
    check   = stream( ErrorFile, 'C', 'close' )
  end  /* --- end of untested code --- */

check = stream( InFile, 'C', 'close' )

return

NotReady:
  if LineNr = 0 then
    say 'NotReady' InFile

return
-------------------- end of wgetmir.cmd --------------------

  Greetings   -=Eddy=-        email: eddy.thilleman@net.hcc.nl

... Windows NT: Neanderthaler Technology
--- GoldED/2 3.0.1
 * Origin: Windows98 is a graphic DOS extender (2:500/143.7)
2433/225

+----------------------------------------------------------------------------+

From: MIKE RUSKAI                                       29-Sep-99 13:08:00
  To: EDDY THILLEMAN                                    01-Oct-99 05:53:17
Subj: hstart wget?

Some senseless babbling from Eddy Thilleman to Mike Ruskai
on 09-27-99  21:45 about hstart wget?...

 ET> Hello Mike,

 ET> 24 Sep 99 13:15, MIKE RUSKAI wrote to EDDY THILLEMAN:
 
 MR> You shouldn't depend on this.  Opening files should always be done
 MR> with the stream() function (or with a Stream object in OREXX),

 ET> I would like to know why.

I explained in another message that it's only a side-effect, not REXX
design (as far as I know).
 
 MR> like this:
 MR> check=stream(InFile,'c','open read') if check='' then /* couldn't
 MR> open the file, act appropriately */

This is actually wrong.  stream() returns 'READY:' or 'ERROR:??' for
opening a file.  It's 'query exists' that returns a null string.

 ET> So I changed my REXX file into this:

 ET> ------------------- begin of wgetmir.cmd -------------------
 ET> /* start wget mirroring each website/file listed in wget.mir */
 ET> arg InFile

 ET> if InFile = '' then
 ET> InFile = 'wget.mir'

 ET> dir = directory( 'c:\www' )
 ET> '@set wgetrc=c:\www\wget.cfg'

 ET> cmd1 = 'wget.exe -m -c -np --tries=1 -k '
 ET> cmd2 = ' -a '
 ET> LogFile = "wget.mir.log"
 ET> filler = copies('-',30)

 ET> signal on notready

 ET> LineNr = 0

 ET> check = stream( InFile, 'c', 'open read' )
 ET> if check != 'READY' then  /* couldn't open the file, act appropriately
 ET> */ do

It's 'READY:', not 'READY'.  Though in PC-DOS 7's REXX, it is 'READY'.  So,
you might want to do this:

if left(check,5) != 'READY' then do

 ET> say "Cannot open file" InFile check
 ET> ch = SysGetKey( NoEcho )
 ET> end
 ET> else
 ET> do while stream( InFile, 'state' ) = 'READY'

lines() or chars() would be better here.

Mike Ruskai
thannymeister@yahoo.com


... Existence is not only temporary, but also pointless.

___ Blue Wave/QWK v2.20
--- Platinum Xpress/Win/Wildcat5! v3.0pr2
 * Origin: FIDO QWK MAIL & MORE!  WWW.DOCSPLACE.ORG (1:3603/140)
267/200
45

+----------------------------------------------------------------------------+

From: Eddy Thilleman                                    29-Sep-99 14:55:15
  To: John Clarke                                       01-Oct-99 05:53:17
Subj: hstart wget?

Hello John,

25 Sep 99 09:02, John Clarke wrote to Eddy Thilleman:

JC> This puts the files in d:\down-dos or d:\down-win etc and writes the
JC> output to screen.  I haven't found a way to put it to screen and to a
JC> logfile at the same time - it seems to be one or the other.

I asked this in the wget mailing list, it is possible if you use a program
'tee' in addition:
pipe the output of wget to tee which will sent that output to two filehandles
(one can be the screen and one can be a log file). I'll figure out how to do
it precisely (wether or not to include the -a logfile parameter) and post the
command line needed. I found a gnu tee version which is in the gnusutil
archive and this one also needs gnushu libs which aren't included in the
gnusutil archive, so I'll have to search for this too (or use a tee version
which doesn't need the gnushu lib files).

  Greetings   -=Eddy=-        email: eddy.thilleman@net.hcc.nl

... Deja moo: I've seen that cow before
--- GoldED/2 3.0.1
 * Origin: Windows95 is a graphic DOS extender (2:500/143.7)
2433/225

+----------------------------------------------------------------------------+

+============================================================================+
