[Date Prev] [Date Next] [Thread Prev] [Thread Next] Date Index Thread Index Search archive:
Date:Mon Apr 30 12:56:04 2003 
Subject:Re: Vector arguments to external procedures 
From:davidy 
Volume-ID:1030430.02 


A while back I asked about how to pass the address of an element of a
vector to an external procedure. My thanks to Roger, Jon and Aaron for
helpful replies.

Here is a rather long summary, for anyone interested, of how I see the
problem now. I'd again be grateful for any further suggestions relating
to the new set of problems and ideas towards the end.

Getting the address
-------------------

There were 3 suggestions:

1. Roger suggesting using fast_subscrintvec to get the base address, and
offsetting it in ordinary Pop. This is the kind of evil trick I was
looking for originally. (I knew it existed, but couldn't remember it.)
Roger also pointed out the garbage collection implications, which turn
out to be the main issue - see below.

2. Jon suggested using an external procedure to offset the address of
the vector, returning it to Pop. I don't know why I hadn't thought of
this, but I hadn't. (I proposed an external interface procedure between
Pop and each real procedure.)

3. Aaron suggested using sysFIELD to get the address. I translated this
into the following code:

    define lconstant vaddr(type);
        ;;; Supports e.g. intvec_addr(n, vec) -> exptr
        ;;; Address of n'th element of vector v (type not checked).
        pop11_need_nextitem("(") -> ;
        pop11_comp_expr_seq_to(")") -> ;
        sysFIELD(false, conspair(type, 1), false, 8:1000);
    enddefine;

    define syntax intvec_addr = vaddr(% "int" %); enddefine;
    ;;; etc.

I'm inclined to use method 3. It returns an exptr struct containing the
address, which I guess creates a little garbage, but avoids its having
to masquerade as an integer and so get translated between a Pop
(big)integer and a machine integer/address. (The external call interface
accepts an exptr struct and automatically passes its contents to the
external procedure, so it's quite neat: exacc foo(intvec_addr(i, v)).)

Dealing with garbage collections
--------------------------------

Thanks for pointing this issue out Roger.

The problem is that if the address is found in user-level Pop, there
might be a GC before it gets passed to the external procedure. This
applies to all three methods above.

Since external argument handling is done in assembly (aextern.s) there
isn't much chance of dealing with it by modifying the system. So the
question is whether I can protect the code from the garbage collector.
I'm not sure I can, but here are some ideas:

1. Call sysgarbage first to get it out of the way. No good! (a) There
might be another one anyway. (b) Big overhead. (c) Unnecessary GCs
destroy my very effective garbage reduction strategy (see
http://www.cs.bham.ac.uk/research/poplog/popvision/help/oldarray ).

2. Use fixed-address vectors. Unsatisfactory. (a) I don't want the
calling program to have to do anything special. (b) Large fixed-address
structures will make for inefficient use of memory.

3. Tinker with the system parameters to try to defer a GC - e.g.
increase popmemlim temporarily. I don't know whether I can make this
totally reliable - does anyone else?

4. Guard the code with sys_lock_heap and sys_unlock_heap. This is no
good as it stands as sys_unlock_heap undoes *all* previous calls to
sys_lock_heap. However, a modification is promising:

    sys_lock_heap();
    exacc ext_foo(intvec_addr(n, v)) =>
    pop_heap_lock_count - 1 -> pop_heap_lock_count;

In practice, this seems to work. One problem is that pop_heap_lock_count
is not documented, though the source and simple tests suggest that
decrementing it undoes the last sys_lock_heap. There's also an overhead
from locking and unlocking the heap - what it does isn't trivial, I
think. Another concern is that one might get an unnecessary
run-out-of-memory mishap while the heap is locked. This *looks*
unlikely, but I'm not sure how to analyse it - does anyone have any
thoughts?

5. Use pop_after_gc to exit to the caller if a GC occurs during the
critical code, then try again if the external procedure had not run
before the GC happened. I think this would be a good strategy (low
overhead unless a GC occurs) except for one thing: I don't know how to
tell in general whether the external procedure had already run when the
GC happened. (If it had already run, it mustn't be run again because it
might have deliberately overwritten its input data.) I feel as if this
ought to be trivial - but in practice I'm stuck.

Overall, I'm beginning to wonder whether to fall back on my original
idea, but will probably explore option 4 above further first. Has anyone
any ideas? Comments extremely welcome!

David