The RI stack again

[pjw]

Handling the arithmetic stack, or RI stack, has been fraught for toolkit writers since the dawn of the QL age. Myths abound, rumours swirl, secret information is whispered among the illuminati, but nothing seems right. Or rather it seems to work for a while, and may then be included in some larger project and forgotten about.

Suddenly you experience mysterious crashes. Programs that used to work start misbehaving and all is chaos as you frantically try to debug millions of line of code before finally succumbing, ending up as a gibbering wreck at some institution for the seriously insane. (Yes, there must be thousands of such individuals dotted around the globe in secure institutions. Where else have all the QLers disappeared to?)

Such a fate need not be yours! There is some literature on the subject. Perhaps survivors could list some vetted ones here as I cant recall the names of any off the top of my head. However, be mindful of anything not written within the last 10 years or so! Thats where the gremlins lurk.

Another thing one must do it TEST the code. The first test is that the code produces the right answer, of course - preferably without crashing the system. The next tests are stress tests. They will be different depending on the function or procedure in question. Below are some rough templates of two such possible tests. These look for memory leakage due to a mishandling og the RI stack and some other ills. The first one is for functions:

Code: Select all

100 RANDOMISE
102 m = FREE_MEM: mm = m
104 t = DATE
106 :
108 FOR i = 0 TO 999
110  r = RND(-1 TO 1)
112  t = MYFUN(r)
114  t = 1 + MYFUN(r)
116  t = 1 + MYFUN(r) + 1
118  t = MYFUN(r) + 1
120 :
122  IF m <> FREE_MEM THEN
124  PRINT m - FREE_MEM
126  m = FREE_MEM
128  END IF
130 END FOR i
132 :
134 PRINT 'Time:'! DATE - t
136 PRINT'Mem:'! mm - FREE_MEM
138 :

This will list every time the functions grabs more memory. One such grab might be as expected, but at the end of the run the memory difference should end up as zero or the first amount grabbed.

As I mentioned, these templates may need to be tailored for the specific function to be tested. If, for example 'r' needed to be a string instead of a number, tests should be made for even and odd lengthed strings.

The next test is for procedures. These dont usually suffer from stack problems. But for people who may, for example, have learnt that A1 has a particular value on entry to a function or procedure, and then use that value to perform some other action, eg reserve memory on the RI stack, they may be in for a surprise!

Code: Select all

100 m = FREE_MEM: mm = m
110 t = DATE
120 :
130 ch = FOPEN("nul"):      REMark SMSQ/E only without extra driver
140 FOR i = 0 TO 999
150  MYPROC#ch; 'some text'
160  MYPROC#ch; 'some texts'
170  MYPROC#ch; 'some tex'
180 :
190  IF m <> FREE_MEM THEN
200  PRINT m - FREE_MEM
210  m = FREE_MEM
220  END IF
230 END FOR i
240 :
250 PRINT 'Time:'! DATE - t
260 PRINT'Mem:'! mm - FREE_MEM
270 CLOSE#ch
280 :

It is important to do as many rounds as possible to ensure there are no leaks. But for BBQLs 99 iterations is as much as patience may accept. 9999+ for QPC2 et al. Also the more iterations the more accurate the timings, where that may be of interest. If a proc/fn is suspiciously slow, that could also indicate a problem.

Unfortunately, there are slight differences in behaviour between the various OSes: Qdos, Minerva, and SMSQ/E. Provided one sticks to documented calls, values and offsets, and doesnt go off-piste on the above mentioned myths etc, one should be alright. However, it doesnt hurt to run these tests on all systems a toolkit is designed to work with.

Here endeth the lesson. Please feel free to challenge any mistakes you think I may have made. We have suffered long enough!

[tofro]

In my experience, it is also important to test the fail path (i.e. you're returning an error code) of (especially) FuNctions.
While S*BASIC tends to clean up after you in PROCedures, it doesn't in FuNctions - And most of the memory leaks I found in my own code were exactly there - The success path normally gets tested reasonably well, and because you don't tend to make a function repeatedly fail, or in a loop, those are the sneaky memory leaks that still use to lurk in the background and only show their ugly head in very long QL sessions.

[martyn_hill]

Hi Per and Toby!

A great topic to bring back in to the light...

My RI Stack reference of choice is Norman's exposition here: https://dilwyn.theqlforum.com/docs/articles/stack.zip

Having fallen-foul so often when writing my own toolkit routines, the advice Norman provides in that article has helped unravel many such problems.

I still get it wrong from time to time and turn back to that article to put me straight again...

As a slight aside, another issue I have faced relates to the register usage (i.e. those regs that get smashed) of the various TRAPs and Vectored routines, which often varies between QDOS/Minerva and SMSQe - and not always as per the respective documentation - though the SMSQe manual still gets corrective updates and I find reliable...

[pjw]

In my experience, it is also important to test the fail path (i.e. you're returning an error code) of (especially) FuNctions.
While S*BASIC tends to clean up after you in PROCedures, it doesn't in FuNctions - And most of the memory leaks I found in my own code were exactly there - The success path normally gets tested reasonably well, and because you don't tend to make a function repeatedly fail, or in a loop, those are the sneaky memory leaks that still tend to lurk in the background and only show their head in very long QL sessions.

Thats interesting, because in the documentation (SMSQ/E Guide, section, 9.10) is says "The arithmetic stack is automatically tidied up both after procedures, and after errors in functions." So the fail path should normally take care of itself unless one has been doing some other RI stack manipulation that only affects the fail path.

The first line of the Guide under that chapter (9.10. The Arithmetic Stack Returned Values) makes the fateful assertion "The top of the arithmetic stack is usually pointed to by A1." I guess thats what caught most people out (it did me, at least). Even if one is somewhat autistic and notices the "usually" it is still confusing. It would perhaps have been slightly clearer if it had said "A1 is often used to point to the top of the stack." and then gone on to explain the circumstances when A1 actually does point to the top of the stack, namely after a call to fetch parameters.

[tofro]

In my experience, it is also important to test the fail path (i.e. you're returning an error code) of (especially) FuNctions.
While S*BASIC tends to clean up after you in PROCedures, it doesn't in FuNctions - And most of the memory leaks I found in my own code were exactly there - The success path normally gets tested reasonably well, and because you don't tend to make a function repeatedly fail, or in a loop, those are the sneaky memory leaks that still tend to lurk in the background and only show their head in very long QL sessions.

Thats interesting, because in the documentation (SMSQ/E Guide, section, 9.10) is says "The arithmetic stack is automatically tidied up both after procedures, and after errors in functions." So the fail path should normally take care of itself unless one has been doing some other RI stack manipulation that only affects the fail path.

The first line of the Guide under that chapter (9.10. The Arithmetic Stack Returned Values) makes the fateful assertion "The top of the arithmetic stack is usually pointed to by A1." I guess thats what caught most people out (it did me, at least). Even if one is somewhat autistic and notices the "usually" it is still confusing. It would perhaps have been slightly clearer if it had said "A1 is often used to point to the top of the stack." and then gone on to explain the circumstances when A1 actually does point to the top of the stack, namely after a call to fetch parameters.

I think that "usually" has to be read as "for someone who writes the S*BASIC interpreters". And as we're not doing that, it "usually" doesn't point to anything specific unless you make it point somewhere.... But I must admit it caught me as well, years ago.

[pjw]

<>
I think that "usually" has to be read as "for someone who writes the S*BASIC interpreters". And as we're not doing that, it "usually" doesn't point to anything specific unless you make it point somewhere.... But I must admit it caught me as well, years ago.

[NormanDunbar]

Afternoon All,

First of all, Martyn, thank for reading my stuff! Sadly, with the advent of SMSQ, the bits where I mention A1 being negative, zero or positive are no longer accurate. Please ignore those bits. They are (were?) correct for JS QDOS.

...The first line of the Guide under that chapter (9.10. The Arithmetic Stack Returned Values) makes the fateful assertion "The top of the arithmetic stack is usually pointed to by A1."

In the document to which Martyn mentions, I have clearly documented the fact that:

One of the first problems I had with the maths stack was the fact that I was
always led to believe that on entry to a function, A1.L points to a suitable
value for the top of the maths stack (relative to A6 that is). This
is not the case.

On entry to a function, the only thing you can be sure of regarding the top of
the maths stack, is that the value is held in BV_RIP(A6) and A1 should be
disregarded as a stack pointer until later.

I too fell for the old information (Pennell, QDOS Companion) that "on entry to a function, A1.L is a suitable value for the top of the stack." Took ages to determine that it wasn't my code that was wrong, but the docs!


Cheers,
Norm.