Acceptance testing methodology
It is desirable to have the number of data points be relatively large and uniformly distributed over the domain of the operation being evaluated
- Else critical points, where implementation is flawed, may be missed.
- The size and spatial distribution of values in the Reference Data Test Set is important and is operation dependent.
- Once the Reference Data Test Set is specified it is relatively easy to evaluate the appropriate error metric over the whole set of values and to find the maximum error on the Reference Data Test Set.
- This maximum error is used to determine the level of compliance of a particular implementation.
Levels of acceptance
- A particular implementation should not be required to meet the standard at the highest level if this induces unnecessary complexity and cost penalties.
- In some applications, users may choose to simplify or approximate the formulations to reduce implementation and computational complexity and in particular to reduce computer processing time.
- In doing so, they are willing to accept some degradation in accuracy for a particular application domain.