REF LIBRARY                          John Gibson, John Williams May 1991
                                            Revised John Gibson Aug 1995

        COPYRIGHT University of Sussex 1995. All Rights Reserved

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
<<<<<<<<<<<<<<<<<<<<<                             >>>>>>>>>>>>>>>>>>>>>>
<<<<<<<<<<<<<<<<<<<<<  POPLOG LIBRARY MECHANISMS  >>>>>>>>>>>>>>>>>>>>>>
<<<<<<<<<<<<<<<<<<<<<                             >>>>>>>>>>>>>>>>>>>>>>
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

Libraries are  programs which  provide much  of Poplog's  functionality.
This REF  file provides  information on  the Poplog  library  structure.
where they are stored (including  variable names), the rationale  behind
their  storage,  how  they  are  found  (using  searchlists),  and   the
procedures for manipulating these searchlists. Details are also given on
the compilation procedures used for library files.

         CONTENTS - (Use <ENTER> g to access required sections)

  1   Introduction

  2   Types of Library

  3   Library Directories
      3.1   Note for VMS users
      3.2   Standard Library Directories

  4   Library Names and Library Search Lists
      4.1   Search List Format
      4.2   Standard Pop-11 Search Lists
      4.3   Manipulating Search Lists

  5   Locating Library Files: syssearchpath

  6   Autoloading Pop-11 Library Files

  7   Loading Library Files with uses

  8   Loading Library Files Explicitly

  9   Standard Library Directory Names




---------------
1  Introduction
---------------

The term  library  denotes  a  program  component,  or  set  of  program
components, that  can be  handled  as a  single  named entity.  Much  of
Poplog's functionality is provided in library form. These libraries  may
be loaded into the core Poplog run-time system as desired.

Poplog's libraries are stored in disk files, one library per file. It is
not possible for one file to contain several libraries.

The name  of a  library is  the name  of the  file (as  a string)  which
contains the library program.




-------------------
2  Types of Library
-------------------

There are various types of library:

    (a) compile libraries

    (b) include libraries

Compile libraries generally contain a single procedure definition,  or a
set of related procedure definitions. They are loaded into the  run-time
image by simply compiling the library file.

Include  libraries,   on  the   other   hand,  consist   of   identifier
declarations. When  referenced,  include  libraries  are  not  compiled:
instead, their  contents  are merged  into  the current  compiler  input
stream, as if the  declarations were explicitly  present in the  current
file. The  term  "include" is  derived  from C.  Include  libraries  are
documented fully in HELP * INCLUDE and REF * PROGLIST.

Some  libraries  are  autoloadable.  These  normally  define  a   single
identifier. Poplog  will automatically  load  such libraries  when  that
identifier is first referenced. The method by which this is accomplished
is discussed below (and also in HELP * AUTOLOAD).

Non-autoloadable libraries may define  several related identifiers,  and
are only loaded on explicit request.




----------------------
3  Library Directories
----------------------

Poplog's library files are divided amongst various library  directories.
Each directory groups libraries of a similar status. Roughly speaking, a
library file is assigned to a particular library directory according to

    (a) subsystem   (i.e. Pop-11, Prolog, Lisp or ML)

    (b) type        (i.e. autoloadable, non-autoloadable, include)

    (c) function    (e.g. X Windows Interface libraries, Ved libraries)



3.1  Note for VMS users
-----------------------
In this file, directory names are specified in Unix format, which may be
translated to VMS format as follows:

    UNIX                                VMS
    $usepop/pop/lib/auto/nl.p     >>    usepop:[pop.lib.auto]nl.p

The section Filename Processing in  REF * SYSUTIL discusses Unix to  VMS
filename translation in more detail.



3.2  Standard Library Directories
---------------------------------
Poplog includes the following Pop-11 library directories:

    # pop/lib/auto
    # pop/lib/data
    # pop/lib/database
    # pop/lib/demo
    # pop/lib/flavours
    # pop/lib/include
    # pop/lib/lib
    # pop/lib/proto
    # pop/lib/sun
    # pop/lib/turtle
    # pop/lib/ved
    # pop/lib/ved/term
    # pop/x/pop/lib

and these Lisp library directories:

    # pop/lisp/modules

and these Prolog library directories:

    # pop/plog/auto
    # pop/plog/lib

and these ML library directories:

    # pop/pml/lib


The status and purpose of these libraries is discussed in  DOC * SYSSPEC
and HELP * LIBFILES.




-----------------------------------------
4  Library Names and Library Search Lists
-----------------------------------------

Library names do not contain any directory information. Poplog therefore
needs a mechanism for  mapping library names  to actual file  pathnames.
This mapping is effected using search lists. A search list specifies the
set of directories in  which to look for  a particular library.  Given a
library name,  and  a search  list,  the procedure  syssearchpath,  will
return the full library file pathname.

The file HELP * SEARCH_LISTS  provides a more  tutorial introduction  to
this topic.

Poplog documentation  is  also  accessed via  search  lists.  These  are
discussed in HELP * VEDSYSFILE and REF * VEDPROCS.



4.1  Search List Format
-----------------------
Each element of  a search list  must be  one of the  following types  of
object:

    (a) string

    (b) list

    (c) word or identifier

    (d) procedure

Strings denote explicit directory names (e.g. '$popautolib').

Lists are considered to be annotated directory names. They consist  of a
directory name (the first item in the list), plus additional information
for use by the Poplog editor, Ved.

Words and identifiers are treated as pointers to a "nested" search list.
They may also be indirect references to strings or procedures.

Procedures compute  library  file  pathnames,  given  library  names  as
argument.

The manner in  which syssearchpath  uses these various  types of  search
list component is discussed below.



4.2  Standard Pop-11 Search Lists
---------------------------------
Pop-11 provides the following search lists as standard:


popautolist -> search_list                                    [variable]
search_list -> popautolist
        The set of directories  where Pop-11 autoloadable library  files
        may be found. Its default value is the list:

            [   '$poplocalauto/'
                '$popautolib/'
                '$popvedlib/'
                '$popvedlib/term/'
                '$usepop/pop/lib/database/'
            ]

        Users may extend  or modify this  list; it is  also extended  by
        various  system   libraries   (such   as   LIB * POPXLIB).   See
        HELP * POPAUTOLIST.


popuseslist -> search_list                                    [variable]
search_list -> popuseslist
        The set of Pop-11 library directories to be searched by the uses
        syntax construct. Its default value is the list:

            [   ^(ident popautolist)
                '$poplocal/local/lib/'
                '$popliblib/'
                '$popdatalib/'
            ]

        Users may extend  or modify this  list; it is  also extended  by
        various  system   libraries   (such   as   LIB * POPXLIB).   See
        HELP * POPUSESLIST.

        (Note that the  use of  'ident popautolist' in  the above  means
        that the value  of popautolist is  not frozen into  popuseslist,
        and changes to the former will be picked up in the latter.)


popsyslist -> search_list                                     [variable]
search_list -> popsyslist
        This list is mainly used by the Popc compiler, and contains  the
        set of all  source and  library directories to  be searched  for
        identifier declarations. Its default value is the list:

            [   '$usepop/pop/src/'
                '$usepop/pop/ved/src/'
                ^(ident popuseslist)
            ]

        Users may extend  or modify this  list; it is  also extended  by
        various  system  libraries  (such  as  LIB * POPXLIB,  which  in
        addition to adding the X library directories to popuseslist also
        adds the  X  source  directory  $usepop/pop/x/src/  directly  to
        popsyslist).

        (Note that the  use of  'ident popuseslist' in  the above  means
        that the value of popuseslist is not frozen into popsyslist, and
        changes to the former will be picked up in the latter.)



4.3  Manipulating Search Lists
------------------------------

flatten_searchlist(search_list) -> list                      [procedure]
flatten_searchlist(search_list, want_props) -> list
        Flattens a search list into a list of directory names (strings),
        i.e.  dereferences  any  words   or  identifiers  contained   in
        search_list. If the  optional argument  want_props is  supplied,
        and not false, then  annotated directory names  (2 or 3  element
        lists) are  returned unchanged.  Otherwise, annotated  directory
        names are reduced to simple directory names.

        Note   that   flatten_searchlist   cannot   sensibly   interpret
        procedures in a search list, so these are quietly ignored.


extend_searchlist(item, search_list) -> search_list          [procedure]
extend_searchlist(item, search_list, at_end) -> search_list
        Adds item  to the  library search  list search_list,  unless  it
        already contains  item.  item  may  be  any  valid  search  list
        component  (i.e.  a  string,  annotated  directory  name,  word,
        identifier, or procedure). When testing whether item is a member
        of  search_list,  extend_searchlist  dereferences  pointers   to
        nested search lists as necessary. If  item needs to be added  to
        search_list, it is added to the end of the list if the  optional
        argument at_end is supplied and true;  it is added to the  front
        of the list otherwise. search_list  is returned unchanged if  it
        already contains item.




----------------------------------------
5  Locating Library Files: syssearchpath
----------------------------------------

syssearchpath(search_list, lib_name) -> filename             [procedure]
syssearchpath(search_list, lib_name, bool) -> filename

        Searches the directories in search_list for the library named by
        lib_name, returning the full pathname of the file if successful,
        and false  otherwise.  syssearchpath examines  the  elements  of
        search_list in left-to-right order,  according to the  algorithm
        described below (and also in HELP * SEARCH_LISTS):

            # If the  element E is  a string, it  is considered to  be a
            directory name. If a file called:

                sysfileok(E) dir_>< lib_name

            exists, then the search is terminated, and that filename  is
            returned. Otherwise, if a directory named:

                sysfileok(E) dir_>< 'doc_index'

            exists, that directory is considered  to be an index to  the
            directory E. The  procedure sys_search_doc_index  (described
            in HELP * MKREFINDEX) is  used to  search the  index for  an
            entry for lib_name, as follows:

                sys_search_doc_index(
                    lib_name, E, if bool then 2:01 else 2:11 endif)

            (bool is an optional argument, which defaults to false).
            If one or more index entries for lib_name are found, then:

                  # If  bool is  false, or  there is  only one  matching
                  entry, the filename  associated with  lib_name by  the
                  index is returned.

                  # If bool is true, and there is more than one matching
                  entry, a  vector containing  the matching  entries  is
                  returned.

          # If the element E is a procedure, it is applied to  lib_name.
            The result,  if  non-false, is  considered  to be  the  full
            pathname of lib_name, and returned.

          # If the element E is a  word or identifier, it is  considered
            to be a pointer to a "nested" search list, which is examined
            recursively.

          # If the  element E  is a  list,  it is  considered to  be  an
            annotated directory  name. The  head of  the list  may  be a
            string, procedure, or word/identifier,  and is processed  as
            described in steps 1, 2, and 3 respectively. If this results
            in a pathname being determined, a copy of the list with  the
            full pathname as its first element is returned.

        syssearchpath is used by  vedsysfile (via vedgetlibfilename)  to
        locate system documentation and library source code.




-----------------------------------
6  Autoloading Pop-11 Library Files
-----------------------------------

The procedures and variables used to implemement Pop-11 autoloading  are
described below. For a more tutorial discussion, see HELP * AUTOLOAD.


syslibcompile(lib_name, dir_list) -> dir                     [procedure]
syslibcompile(lib_name) -> dir
        This procedure is  used by  sys_autoload to  locate and  compile
        Pop-11 library files.  It searches the  directories in  dir_list
        for the library named by lib_name.  If such a library is  found,
        it is compiled (with pop11_compile), and the relevant  directory
        is returned. Otherwise, syslibcompile returns false.

        The argument  lib_name may  be  a string  or  a word.  The  file
        extension '.p' is added to lib_name if necessary.

        If the  argument  dir_list  is  omitted,  popautolist  is  used.
        dir_list may contain the full range of search list components as
        described above.


prautoloadwarn(lib_name)                            [procedure variable]
        Called by syslibcompile when it has determined the full pathname
        of the library named lib_name, immediately before compiling  it.
        (Note that this only happens when lib_name is found in an actual
        directory, not when 'found' by  a procedure in popautolist.  Its
        default value  is erase,  which means  that autoloading  etc  is
        normally done silently; the  procedure sysprautoloadwarn may  be
        assigned to this variable when notification is required.


sysprautoloadwarn(lib_name)                                  [procedure]
        This  procedure   may  be   assigned  to   prautoloadwarn   when
        notification of autoloading is required. It prints the line

                ;;; AUTOLOADING lib_name

        on the standard output (i.e. through cucharout).


sys_autoload(word) -> dir                                    [procedure]
        This procedure is  called by parts  of the system  that need  to
        check that an autoloadable definition of a permanent  identifier
        called WORD  is  loaded  (e.g.  see  the  description  of  macro
        expansion in REF * PROGLIST, and sysdeclare in REF * VMCODE).

        It first checks to see if word is already defined as a permanent
        identifier, and returns true if so. Here 'defined' means that

                isdefined(word)

        is true, i.e. that word has  an attached identifier that is  not
        just a 'weak' declaration (see REF * IDENT).

        If word is not defined, but pop_autoload is false, then false is
        returned. Otherwise,  if  pop_autoload  is  true,  it  tries  to
        autoload word:  If  word has  an  autoloading procedure  in  the
        property sys_autoload_action, then that is called, i.e. if

                sys_autoload_action(word) -> P

        is a procedure, then

                P(word) -> dir

        Otherwise, the autoloading procedure defaults to

                syslibcompile(word) -> dir

        to try and autoload a file 'word.p' from one of the  directories
        in popautolist. In either case,  the result of the procedure  is
        returned.

        It  should   normally   be   the  case   that   an   autoloading
        procedure/autoloaded  file  will  define  word  as  a  permanent
        identifier; note that  during the call  of syslibcompile or  the
        autoloading procedure, word is marked specially, so that a  call
        of sys_autoload on the same word during compilation of the  file
        will return  false  rather  than  cause  a  recursive  call.  In
        addition, an abnormal exit from sys_autoload during  compilation
        will remove any  declaration that word  may have acquired  (thus
        ensuring  that  word  remains  undefined  after  a   compilation
        mishap).


sys_autoload_action(word) -> p_or_false                      [procedure]
p_or_false -> sys_autoload_action(word)
        Property used by sys_autoload. If word is to be autoloaded, and

                sys_autoload_action(word) -> p

        is  a  procedure,   then  that   will  be   called  instead   of
        syslibcompile.

        (Note that whenever a given word becomes (strongly) declared  as
        a  permanent  identifier,  any  sys_autoload_action  for  it  is
        removed.)


pop_autoload -> bool                                          [variable]
bool -> pop_autoload
        This  variable  (default  value  true)  can  be  used  to   stop
        sys_autoload from autoloading undefined identifiers; when false,
        sys_autoload just returns false for such an identifier.

        Setting pop_autoload false is  particularly useful for  programs
        that wish  to  use  itemread  or nextitem  to  read  words  from
        proglist with macro expansion, but  do not want undefined  words
        to be autoloaded.




----------------------------------
7  Loading Library Files with uses
----------------------------------

See HELP * USES for a description of the Pop-11 uses syntax construct.

uses_lib_idents(lib_name, lib_list, id_names, flags)         [procedure]
        System procedure called by the * uses syntax construct.


useslib(lib_name)                                   [procedure variable]
        This procedure was previously called by the uses syntax word. It
        is now the same as

                uses_lib_idents(lib_name, false, [], 0)




-----------------------------------
8  Loading Library Files Explicitly
-----------------------------------

The following  procedures  and syntax  words  are provided  for  loading
library files explicitly.


subsystem_libcompile(lib_name, dir_list) -> bool             [procedure]
        Loads  the  library  lib_name,  using  its  file  extension   to
        determine to which subsystem the library belongs and hence which
        search-lists and which compiler to use.

        If a subsystem-specific version of the library cannot be  found,
        the given dir_list is searched. The result indicates whether  or
        not the library was found and compiled.


loadlib(lib_name)                                   [procedure variable]
loadlib(lib_name, lib_list)
        Loads the library lib_name, by doing

            subsystem_libcompile(lib_name, lib_list)

        where lib_list defaults  to popuseslist  if not  supplied as  an
        optional second argument.

        An error is signalled if the library file cannot be found,  i.e.
        if subsystem_libcompile returns false.

        (Note that subsystem_compile_warn is locally set to erase during
        this procedure.)


syslibwarning(lib_name)                                      [procedure]
        Uses sysprmessage to print a message of the form

            ;;; LOADING LIB lib_name

        syslibwarning is  the default  value of  the procedure  variable
        libwarning, a call to which is planted by the syntax word lib.


libwarning(lib_name)                                [procedure variable]
        A call to this procedure is  planted by the syntax word lib,  in
        order to inform the user that  the library lib_name is about  to
        be compiled. The  default value of  libwarning is the  procedure
        syslibwarning.




-----------------------------------
9  Standard Library Directory Names
-----------------------------------

The following variables  refer to standard  Poplog library  directories,
via the environment variables (Unix)  or logical names (VMS) defined  in
the file $popcom/popenv (Unix) or POPCOM:POPENV.COM (VMS).


popdisk -> dir                                                [variable]
dir -> popdisk
        The root of the Poplog directory tree. Default value  '$usepop/'
        (Unix) or 'usepop:' (VMS). This variable is only used at system-
        build time; altering its value has no effect whatsoever.


poplibdir -> dir                                              [variable]
dir -> poplibdir
        The main Pop-11  autoloadable library  directory. Default  value
        '$popautolib/' (Unix) or 'popautolib:'  (VMS). This variable  is
        only used at system-build time; altering its value has no effect
        whatsoever.


vedlibdir -> dir                                              [variable]
dir -> vedlibdir
        The main  Ved  autoloadable  library  directory.  Default  value
        '$popvedlib/' (Unix)  or 'popvedlib:'  (VMS). This  variable  is
        only used at system-build time; altering its value has no effect
        whatsoever.


popliblibdir -> dir                                           [variable]
dir -> popliblibdir
        The main  Pop-11  non autoloadable  library  directory.  Default
        value '$popliblib/' (Unix) or  'popliblib:' (VMS). The value  of
        this variable is  inserted into  popuseslist when  that list  is
        first created, after  which, altering  its value  has no  effect
        whatsoever.


popdatalib -> dir                                             [variable]
dir -> popdatalib
        A Pop-11 library directory,  containing miscellaneous data  used
        by other library programs.  Default value '$popdatalib/'  (Unix)
        or 'popdatalib:' (VMS). The value  of this variable is  inserted
        into popuseslist when that list  is first created, after  which,
        altering its value has no effect whatsoever.


popsunlib -> dir                                              [variable]
dir -> popsunlib
        The library  directory  where SUN-specific  Pop-11  software  is
        kept. Default  value  is '$popsunlib/'  (Unix)  or  'popsunlib:'
        (VMS). This variable  is only defined  if running on  a SUN,  in
        which case its value is  included in popautolist (and  therefore
        popuseslist). To define popsunlib on a machine other than a SUN,
        load LIB * POPSUNLIB. (Default value '$popsunlib/' ).


popturtlelib -> dir                                           [variable]
dir -> popturtlelib
        This variable is only defined if the library  LIB * POPTURTLELIB
        has been  loaded. It  holds the  library directory  used by  the
        Pop-11  "turtle"  graphics   package.  Its   default  value   is
        '$usepop/pop/lib/turtle/'  (Unix)   or   USEPOP:[POP.LIB.TURTLE]
        (VMS). Loading LIB * POPTURTLELIB adds  popturtlelib to the  end
        of popautolist.




--- C.all/ref/library
--- Copyright University of Sussex 1995. All rights reserved.
