[Date Prev] [Date Next] [Thread Prev] [Thread Next] Date Index Thread Index Search archive:
Date:Mon May 3 20:43:30 2003 
Subject:Re: Vector arguments to external procedures 
From:Aaron Sloman 
Volume-ID:1030503.02 


steve@watchfield.com writes:

> Date: Sat, 3 May 2003 03:41:29 +0000 (UTC)
>
> Hi David,
>
> >Just out of curiosity, how does the GC know when to collect destroy stuff?
> ...
 
> ...
> The problem David Young has is that the Poplog garbage collector is a
> _relocating_ collector.  In other words the pointer values are
> changed during garbage collection - disrupting his strategy of
> passing pointer offsets.  There is no mechanism in Poplog for
> protecting an object against relocation and it would be technically
> demanding to do such a thing.

I think this is not correct.

First of all, in REF SYSTEM this information is provided:

sys_lock_heap()                                              [procedure]
        Tells the system to 'lock' the heap at its current endpoint. The
        effect of this is  that all structures in  the heap before  this
        point will automatically be treated as non-garbage (considerably
        reducing the amount of work needed  to be done on them  during a
        garbage collection). Structures created after the  sys_lock_heap
        will be treated normally.

I think that implies that items in the locked portion of the heap will
not be re-located during a garbage collection, as implied in some
of the earlier discussions on this topic.

More importantly, REF EXTERNAL_DATA in section 8 addresses this problem.

It was essential to deal with this for the X interface to work properly
since that allows pop-11 structures, e.g. callback procedures, strings,
or vectors of strings in the case of scrolling text widgets, to be given
to external (C) programs provided by the X libraries.

For people without Poplog, the REF file is accessible here:

    http://www.cs.bham.ac.uk/research/poplog/doc/popref/external_data

Here's the introduction to section 8:

8  Fixed-Address Poplog Structures for External Use
---------------------------------------------------

The sections above deal with the representation of external data  inside
Poplog; this section considers the use of native Poplog data  structures
externally, i.e,  when  passed  to external  functions  etc.  The  major
problem likely to arise in this context concerns garbage collection, for
the following reasons:

    (1) The Poplog garbage collector will discard (and reuse the  memory
        occupied by)  any  structure  which  has  no  references  to  it
        remaining inside Poplog;

    (2) When  an  (ordinary)  structure  is  retained  by  the   garbage
        collector, it may be moved to  a new location (i.e. its  address
        may change);

    (3) The garbage collector has no  knowledge of references to  Poplog
        structures stored by external functions, and so cannot take them
        into account when deciding  to retain structures  in (1), or  to
        correct the addresses they refer to in (2).

However, since garbage collections can only occur while executing Poplog
code, the  above present  no  problem PROVIDING  the external  use  of a
structure always finishes before control is ever returned to Poplog.

    The problem  occurs only  when  an external  agent  in some  way  or
another retains a structure during a return to Poplog execution  (during
which a garbage collection happens), and then later expects to use  that
structure again (at which time the externally-stored reference may  have
become invalid,  either  through  (1)  or (2)).  That  is,  the  control
sequence

?       external code  ??>  Poplog  ??>  external code
?                         (GC caused)

must occur  (either  by returning  to  Poplog and  then  re-calling  the
external function,  or  by  calling-back to  Poplog  from  the  external
function and then returning from the callback, etc).

    While this scenario will  not arise in  many programs (e.g.  because
data is always passed anew to an  external function on each call, so  it
never  relies  upon  stored  references),  in  other  programs  it  will
difficult or impossible to avoid.  Poplog therefore provides a  solution
to (2)  by making  possible the  creation of  fixed-address  structures,
whose memory locations do not change;  this is done with the  procedures
described below.  In  all  cases,  they  call  an  ordinary  constructor
procedure, but in such  a way that the  result is allocated memory  in a
separate part  of the  heap reserved  for fixed-address  objects --  and
where  its  address  is  guaranteed  to  remain  unchanged  by   garbage
collection.

    However, while  this  solves  (2),  the  potential  problem  of  (1)
remains: that  is, external  code  could have  stored references  to  an
(albeit fixed-address)  structure  for  which  there  are  no  remaining
references inside  Poplog  itself (and  so  a garbage  collection  would
destroy the  object). The  procedures below  therefore also  provide  an
option to automatically hold the structures they create on a global list
(from which,  when  no longer  required,  they  can be  freed  with  the
procedure free_fixed_hold). A  given program only  need use this  option
for structure(s) for which it does not anyway retain suitable references
itself (e.g. in variables or other data structures).

=======================================================================

Further details are in the file, describing procedures cons_fixed,
init_fixed, copy_fixed, is_fixed, free_fixed_hold, sys_grbg_fixed
and others.

WARNING: I do not recommend mixing fixed address structures (e.g. X
window utilities, widgets, etc. ) with the use of sys_lock_heap and
sys_unlock_heap. My impression is that once you have used a fixed
address structure after locking the heap, if you then unlock the heap
you cannot re-use the previously locked space before the fixed
structure.

I formed this impression after repeatedly locking and unlocking the heap
using X facilities and found the the heap size kept growing, whereas
if I did not lock and unlock it remained more constant.

But I have not repeated this experiment recently.

Aaron



>
> The root cause of the difficulty is that pointer offsets are not a
> recognized data type in Poplog.  This is a long standing omission of
> a "well-known" [*] requirement.  The result is that there is no
> representation of external pointer offsets either.
>
> [*] "Well-known" in this context means that you have put up with one
> of my extended rants about garbage collection.  Other "well-known"
> requirements include cooperation with manual deallocation, the
> ability to perform speculative garbage collection, a key-specific
> post-GC hook, pointer elision, pointer replacement, pointer reversal,
> and proper monitoring tools.
>                     
> --                    
> Steve                   
====
Aaron Sloman, ( http://www.cs.bham.ac.uk/~axs/ )
School of Computer Science, The University of Birmingham, B15 2TT, UK
EMAIL A.Sloman AT cs.bham.ac.uk   (ReadATas@please !)
PAPERS: http://www.cs.bham.ac.uk/research/cogaff/ (And free book on Philosophy of AI)
FREE TOOLS: http://www.cs.bham.ac.uk/research/poplog/freepoplog.html