Achievement and Quality: Higher Education in the Arts

Numbers and Evaluation in the Arts: Critical Questions

Questions

Given the issues presented above, how can individuals and groups evaluating the arts, and particularly the preparation of professional artists use numbers, quantification, and counting effectively and respect the integrity of how the art form and education in it produce results?  The questions below are offered as one possible way to determine the role of numbers in any particular evaluation project.

1. What do you want to know? 
2. What is the scope of what you want to know?   How comprehensive, how much breadth, how much depth, what level of sophistication?
3. Why do you want to know it?   For example, because you want to, need to, or must?
4. Given the answers to the first three questions,
   
   a.  what roles can numbers, quantification, and counting play? 
   b.  what will any proposed use of numbers reveal?
   c.  what will they not reveal?
   d.  what may they obscure?
   e.  how may they be misinterpreted or misused?
   f.   how easily can they be contested as “proof?”
5. Given the answers to the first four questions, what is the relative importance of quantitative data, qualitative information, and personal judgment in determining what you want to know at the level you want to know it?
6. What is the time and other resource costs of obtaining the numbers-based information you want to use?  To what extent are these costs worth the benefits expected?
7. How do your proposed decisions about use of numbers relate to the issues and possible dangers raised in Boyle’s Counting Paradoxes?
8. How can you minimize risks if you must evaluate using numbers in ways that are inappropriate or potentially misleading?
9. Once you have the numbers you want, how can you best present what they actually mean, and if appropriate, what they do not mean?  How can you best relate them to other information and judgment?

More Information

• Boyle's Counting Paradoxes
• Resources Regarding Numbers and Evaluation in the Arts
  

 _____________________________

Kindly report any Web site problems or broken links to Willa Shaffer (wshaffer@arts-accredit.org).

Thank you.