[Date Prev] [Date Next] [Thread Prev] [Thread Next] Date Index Thread Index Search archive:
Date:Mon Feb 6 15:08:45 1998 
Subject:Re: Memoization in pure languages? 
From: Robin Popplestone  
Volume-ID:980206.01 

I thought to go back to the original published material on the subject.
Extracting from:

Michie D, "'Memo' Functions and Machine Learning" From Nature, Vol 218,
April 6 1968.

Michie begins by setting the idea in the context of machine-learning, with
particular reference to Samuel's checkers program. The most definitive
text is

  The present proposals involve a particular way of looking at functions.
  This point  of view asserts that a function is not to be identified with
  the operation by which it is evaluated (the rule for finding the
  factorial, for instance) nor with its representation in the form of a
  table or other look-up medium (such as a table of factorials). A function
  is a function, and the means chosen to find its value in a given case is
  independent of the function's intrinisic meaning. This is no more than a
  retatement of a mathematical truism, but it is one which has been lost
  sight of by the designers of our programming languages. By resurrecting
  this truism we become free to assert: (1) that the apparatus of evaluation
  associated with any given function shall consist of a "rule part"
  (computational procedure) and a "rote part" (lookup table); (2) that
  evaluation in the computer shall on each given occasion proceed whether
  by rule, or by rote, or by a blend of the two, solely as dictated by the
  expediency of the moment; (3) that the rule versus rote decisions shall be
  handled by the machine behind the scenes; and (4) that various kinds of
  interaction be permitted to occur between the rule part and the rote part.

Michie elaborates on (4),

  Thus each evaluation by rule adds a fresh entry to the rote.

Which is, I think close to the current understanding of memoisation, but
adds

  The rule itself may be self-modifying and seek to increase its
  agreement with the contents of the rote. [which may have been
  set up or modified independently from the operation of the rule].

So, from Michie's perspective, something like data-mining (at least as
incorporated in Clementine) is also an instance of memoisation.

There follows  a discussion  of a  humanised scenario  in which  a  person,
needing, on call,  to evaluate a  mathematical function (hcf)  from a  rule
could improve his performance by constructing a card-index (the rote).

He then discusses the effect of integrating a rote into the evaluation of
recursively defined functions.

  I shall suppose that a "card index" regime for the evaluation of
  functions is available, so that we have facilities for converting
  functions into "memo-functions". Because this can only be done at all
  easily using an open-ended programming language free of arbitrary
  constraints, I shall conduct my illustration inf POP-2 which has the
  further advantage that the needed facilitiy is actually available
  in the form of a half a dozen POP-2 library routines.

  We define the rule part of factorial as follows:

        function fact n;
            if n<0 or if not (n.isinteger) then undef else
            if n = 0 then 1 else n * fact(n-1) close
            end

[there are in fact two syntax-errors here - the "if" before the "not" is
erroneous - the "else <newline> if" should be
"elseif". In POP-2 "n.isinteger" was a syntactic variant of isinteger(n).
"close" means "end if".]

Michie continues:

    To endow fact with the "memo" facility, using Popplestone's routines,
    we merely write

        newmemo(fact,100,nonop=) -> fact;

    This replaces the old definition of fact with a new one with a memo
apparatus attached, such that the rote has an upper fixed limit of 100
entries and uses the "=" relaation for look-up purposes.

[nonop in POP-2 was the way of passing an infix as a parameter, and the
"->" notation is an assignment  value -> variable].

My own understanding of what was originally intended when the "memo" term
was coined was that the specification of the rule should be orthogonal
to its integration with the rote. Within the quasi-function paradigm of
POP-2 I would claim that we did a reasonable job of realising this.

So I would see orthogonality as compromised when    Torben Mogensen
proposes:

> You don't need explicit state to do memoization in a functional
> language. As an example, we can define a memoizing fibonacci function
> by using a list:

>fib :: Int -> Int
>fib 0 = 0
>fib 1 = 1
>fib n = fib_list !! (n-1) + fib_list !! (n-2)
>
>fib_list = map fib [0..]

> Laziness ensures that only the needed elements of the list will be
> computed. You can avoid the linear search through the list by using an
> array instead.

So, for a pure functional language, I see memoisation as necessarily a
system capability and not something a user can write, though of course if
user-specified transformation was on the menu (as urged by Tim Sheard) one
can see that orthogonality could be achieved that way.

Incidentally, in a discussion in Glasgow with Simon Peyton-Jones and
Cordelia Hall, it was pointed out to me that memoisation could affect
semantics. This it clearly would do if you memoised a non-strict function,
such as a constructor. [Lisp and its descendents to this with symbols].

Robin.