[Date Prev] [Date Next] [Thread Prev] [Thread Next] Date Index Thread Index Search archive:
Date:Mon Oct 17 07:34:47 1994 
Subject:Error Message Questionnaire (Re-Post) 
From:Andy Marks 
Volume-ID:941018.01 

Error Message Questionnaire
===========================

Greetings,

--
NOTE: This questionnaire has been previously posted to this
newsgroup this year.  If you have already replied, there is
no need to do so again.  The re-posting is mainly for those
people who missed the first posting and US students who were
on summer holidays during the first posting.
--

Here is your big chance to have a say about the quality of
those nasty little error messages that prevent us from
accomplishing our favorite task - programming! I am using
the results from this questionnaire in my research (for a
Masters by Research degree at Monash University) into error
messages and have posted this message to all Internet
newsgroups dealing with programming languages.

Although this posting might look quite large, relax - there
are only 6 questions in total, and the final one is complet-
ely optional: most of the questions are just picking options
from a list anyway, so it shouldn't take more than 10-15
minutes to complete.

If you would like your response to remain confidential,
please indicate this - otherwise you will be given a
"thanks" somewhere in my thesis (best I can do, sorry!).

Please email replies back to the original sender. If you are
NOT replying to this questionnaire through the news reader,
address your reply to "amarks@nellads.cc.monash.edu.au".

Also, please mark all answers on the original copy of the
questionnaire - as opposed to writing your answers in a
freeform manner, which will make collating the results a lot
more difficult and time-consuming.

When all responses have been collected, a summary of the
results will be posted back to each newsgroup. Then we'll
all be able to sleep safely with the knowledge that we all
make mistakes, and no-one is perfect! :-)

-----------------------------------------------------------

Section A: PROGRAMMING LANGUAGE BACKGROUND - these questions
will help me learn a little about the language you're
referring to in your answers. The term "this language"
refers to the one served by the newsgroup this questionnaire
appeared on. For example, if you got this questionnaire from
"comp.lang.apl", then "this language" refers to apl.

1a.Which programming language(s) are you referring to in
   your answers?

   ada              dylan          ml               rexx
   apl              eiffel         modula2          sas
   asm              forth          modula3          sather
   awk              fortran        oberon           scheme
   basic            functional     objective-c      sigplan
   visual-basic     hermes         pascal           smalltalk
   c                icon           perl             tcl
   c++              idl            pop              teco
   cobol            intercal       postscript       verilog
   cfutures         lisp           prograph         vhdl
   clos             logo           prolog           visual
   clu              misc           python

(*)   other: 
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
            insert other language here, if applicable

   (please mark ONE or MORE of these languages using a "*")

b. If applicable, please indicate below the particular
   environment(s) of this language you use (for example,
   Borland C++ versus Microsoft C++).

(*)   environment: 
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                  insert environment name(s) here

-----------------------------------------------------------

Section B: PROGRAMMER BACKGROUND - these questions will give
me some idea of your background and level of experience.
Programmer experience and background is reputedly one of the
most influential factors when dealing with error messages.
The term "this language" refers to the one selected in question
one.

2a.How many years have you been actively programming in this
   language?

       less than one year
       1-2 years
       3-5 years
       5-10 years
       more than ten years

   (please mark ONE of these year ranges with a "*")

NOTE: By "actively programming", I mean consistent use of a particular
language on a regular basis.  For example, usage for at least an hour
each day or a couple of hours each week, say, in the regime of a
course/degree/job of some sort.

b. Using the levels of experience below, how experienced
   would you rate yourself in this language?

       novice programmer
       intermediate programmer
       expert programmer

(*)    other: 
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
             insert other level of experience, if applicable

   (please mark ONE of these levels with a "*")

NOTE: A "novice" programmer is still learning the language,
needs to refer to manuals frequently and is often
inefficient in using the language. An "intermediate"
programmer is comfortable with the language and is familiar
with most of/all of the commands of the language. An
"expert" programmer uses the language in the most efficient
way possible and spends time searching for better ways to
use the language.

3. Which other languages have you used? Please also indicate
   your level of experience in each language you've used by
   marking each one with a number indicating the number of
   years you have been actively using the language.  For 
   example, "3 c" would indicate three years experience in
   the C language.

   ada              dylan          ml               rexx
   apl              eiffel         modula2          sas
   asm              forth          modula3          sather
   awk              fortran        oberon           scheme
   basic            functional     objective-c      sigplan
   visual-basic     hermes         pascal           smalltalk
   c                icon           perl             tcl
   c++              idl            pop              teco
   cobol            intercal       postscript       verilog
   cfutures         lisp           prograph         vhdl
   clos             logo           prolog           visual
   clu              misc           python

(*)   other: 
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
            insert other language here, if applicable

   (please mark ONE or MORE of these languages with a number)

-----------------------------------------------------------

Section C: ERROR MESSAGES - these questions are the most
important part of the questionnaire; they will tell me how
you rate the quality of the error messages in this language.
By "error messages", I mean the messages generated by the
compiler/interpreter when your code is being compiled or
interpreted (probably due to a syntax violation), or at
run-time.

4. What is your general impression of the quality of error
   messages in this language?

       poor
       below average
       average
       good
       excellent

   (please mark ONE of these levels of quality with a "*")

5. Using the kinds of problems below, how do you see the
   error messages in this language being deficient?

       obscure/vague in content
       misleading in information
       unfriendly in tone
       unhelpful in removing error
       jargonistic in wording
       accusatory in tone

(*)    other:
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
             insert other problem, if applicable

   (please mark NONE/SOME/ALL of these problems with a "*")

-----------------------------------------------------------

Section D: OPTIONAL - only answer this question if you wish
to and can remember the wording of some of the error
messages in this language.

6. What are some of the most common error messages you
   encounter in this language? (also indicate whether they are
   satisfactory/unsatisfactory, and how you would improve them)

(>) --- (please put your answer below here)

(<) --- (please put your answer above here)

NOTE: The term "satisfactory/unsatisfactory" in the above
question refers to your opinion as to whether the error
message clearly indicates the source of the error and how to
remove the error.

-----------------------------------------------------------

Thank you for the time and effort spent in making your
contribution.