Keywords: dejean02a

Next: Small Disjuncts are beneficial: Up: First Experiment Previous: Evaluation:

#### The effect of refinement:

We can immediately see the positive effect of the refinement mechanism. The last two rows provide the F1-score of the system for different parameters values without and with the use of exception rules. Refinement improves all results, the F1-score gaining between 1 and 2.5%, which is a very good increase for this task. More interestingly, refinement is especially efficient for since the gain is +2.49 and then provides the best F1-score, outperforming all the other results without refinement. Results deteriorate for higher values of (0.90). This may be due to the fact that is too close or greater than the noise level. One other interesting result is the flattening of the F1-scores. Without refinement rules, the variation of F1-score is much significant and more correlated to . The use of refinement lessens these variations. This result meets our expectation. We can then conclude that refinement is very helpful since it improves results uniformly and gives less importance. This is an interesting issue since such a threshold is a major parameter of inductive systems and has a significant impact on results.

In traditional top-down induction systems, a high value of provides a high precision, but can provide a poor recall. Here, precision and recall are always close one another, even for ``extreme'' values such as 0.5 (P:90.72, R:90.69) or 0.9 (P:88.67, R:89.20). Without refinement the difference between precision and recall increases: P:87.50, R:88.94 for . To increase the size of the context (3 elements) marginally improves results.

The value =0.75 provides the best tradeoff between accuracy and the number of rules (and also the best result without using refinement). When is low (for example 0.5), exceptions of exceptions appear and even exceptions of exceptions of exceptions. We have never observed a recursion level greater than 3. Of course, the lower the value of , the higher the number of exceptions. When exceptions represent 16% of the rules, when , they represent 60%.

Next: Small Disjuncts are beneficial: Up: First Experiment Previous: Evaluation: