leila.evaluate
Class Evaluate

java.lang.Object
  java.awt.Component
      java.awt.Container
          java.awt.Panel
              java.applet.Applet
                  javax.swing.JApplet
                      leila.evaluate.Evaluate

All Implemented Interfaces:

java.awt.image.ImageObserver, java.awt.MenuContainer, java.io.Serializable, javax.accessibility.Accessible, javax.swing.RootPaneContainer

public class Evaluate
extends javax.swing.JApplet

This class is part of LEILA (http://mpii.de/yago-naga/leila). It is licensed under the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0) by the author Fabian M. Suchanek (http://suchanek.name).

Evaluate.java compares the output pairs of a system (e.g. LEILA) to ideal pairs. The ideal pairs for file X.lgo should be in a file X.xxx (where xxx is an arbitrary but fixed extension). This file should contain lines of the following form:

% Optional comments
first word / second word
first word / second word
...

This is the output produced by HandTag.java.

See Also:

Serialized Form

Nested Class Summary

static class

Evaluate.IESystem A list of supported systems

Nested classes/interfaces inherited from class javax.swing.JApplet

javax.swing.JApplet.AccessibleJApplet

Nested classes/interfaces inherited from class java.applet.Applet

java.applet.Applet.AccessibleApplet

Nested classes/interfaces inherited from class java.awt.Panel

java.awt.Panel.AccessibleAWTPanel

Nested classes/interfaces inherited from class java.awt.Container

java.awt.Container.AccessibleAWTContainer

Nested classes/interfaces inherited from class java.awt.Component

java.awt.Component.AccessibleAWTComponent, java.awt.Component.BaselineResizeBehavior, java.awt.Component.BltBufferStrategy, java.awt.Component.FlipBufferStrategy

Field Summary

double[]	array Holds an array of precision/recall-values [p1,r1,p2,r2,...]
static java.util.regex.Pattern	cvPattern Pattern of CV output pairs
static java.util.regex.Pattern	cvSimplePattern Pattern of CV output pairs in their simple form
static java.util.Set<java.lang.String>	files Holds the files for which ideal pairs exist
static java.util.List<EvaluatePair>	idealPairs Holds ideal pairs
static java.util.regex.Pattern	leilaPattern Pattern of LEILA output pairs
static java.util.List<EvaluatePair>	outputPairs Holds output pairs
static java.util.regex.Pattern	text2OntoPattern Pattern of text2onto output pairs
static java.util.regex.Pattern	textToOntoPattern Pattern of textToOnto output pairs
static InstanceOf	wordNet Holds a WordNet for approximate comparisons

Fields inherited from class javax.swing.JApplet

accessibleContext, rootPane, rootPaneCheckingEnabled

Fields inherited from class java.awt.Component

BOTTOM_ALIGNMENT, CENTER_ALIGNMENT, LEFT_ALIGNMENT, RIGHT_ALIGNMENT, TOP_ALIGNMENT

Fields inherited from interface java.awt.image.ImageObserver

ABORT, ALLBITS, ERROR, FRAMEBITS, HEIGHT, PROPERTIES, SOMEBITS, WIDTH

Constructor Summary

Evaluate(double[] a)
Constructor from an array of precision/recall-values

Method Summary

static void	addIdeal(java.io.File f) Adds the ideal pairs from a given file.
static void	addOutput(java.io.File f, Evaluate.IESystem system) Adds the outputPairs pairs from a given result file.
static double[]	eval(int approx, boolean sym) Calculates precision and recall.
static double[]	evalIdeal(int approx, boolean sym) Calculates precision and recall according to the ideal metric.
void	init() Does nothing
static void	main(java.lang.String[] args) User interface
void	paint(java.awt.Graphics g) Draws the precision/recall-dots

Methods inherited from class javax.swing.JApplet

addImpl, createRootPane, getAccessibleContext, getContentPane, getGlassPane, getGraphics, getJMenuBar, getLayeredPane, getRootPane, getTransferHandler, isRootPaneCheckingEnabled, paramString, remove, repaint, setContentPane, setGlassPane, setJMenuBar, setLayeredPane, setLayout, setRootPane, setRootPaneCheckingEnabled, setTransferHandler, update

Methods inherited from class java.applet.Applet

destroy, getAppletContext, getAppletInfo, getAudioClip, getAudioClip, getCodeBase, getDocumentBase, getImage, getImage, getLocale, getParameter, getParameterInfo, isActive, newAudioClip, play, play, resize, resize, setStub, showStatus, start, stop

Methods inherited from class java.awt.Panel

addNotify

Methods inherited from class java.awt.Container

add, add, add, add, add, addContainerListener, addPropertyChangeListener, addPropertyChangeListener, applyComponentOrientation, areFocusTraversalKeysSet, countComponents, deliverEvent, doLayout, findComponentAt, findComponentAt, getAlignmentX, getAlignmentY, getComponent, getComponentAt, getComponentAt, getComponentCount, getComponents, getComponentZOrder, getContainerListeners, getFocusTraversalKeys, getFocusTraversalPolicy, getInsets, getLayout, getListeners, getMaximumSize, getMinimumSize, getMousePosition, getPreferredSize, insets, invalidate, isAncestorOf, isFocusCycleRoot, isFocusCycleRoot, isFocusTraversalPolicyProvider, isFocusTraversalPolicySet, layout, list, list, locate, minimumSize, paintComponents, preferredSize, print, printComponents, processContainerEvent, processEvent, remove, removeAll, removeContainerListener, removeNotify, setComponentZOrder, setFocusCycleRoot, setFocusTraversalKeys, setFocusTraversalPolicy, setFocusTraversalPolicyProvider, setFont, transferFocusBackward, transferFocusDownCycle, validate, validateTree

Methods inherited from class java.awt.Component

action, add, addComponentListener, addFocusListener, addHierarchyBoundsListener, addHierarchyListener, addInputMethodListener, addKeyListener, addMouseListener, addMouseMotionListener, addMouseWheelListener, bounds, checkImage, checkImage, coalesceEvents, contains, contains, createImage, createImage, createVolatileImage, createVolatileImage, disable, disableEvents, dispatchEvent, enable, enable, enableEvents, enableInputMethods, firePropertyChange, firePropertyChange, firePropertyChange, firePropertyChange, firePropertyChange, firePropertyChange, firePropertyChange, firePropertyChange, firePropertyChange, getBackground, getBaseline, getBaselineResizeBehavior, getBounds, getBounds, getColorModel, getComponentListeners, getComponentOrientation, getCursor, getDropTarget, getFocusCycleRootAncestor, getFocusListeners, getFocusTraversalKeysEnabled, getFont, getFontMetrics, getForeground, getGraphicsConfiguration, getHeight, getHierarchyBoundsListeners, getHierarchyListeners, getIgnoreRepaint, getInputContext, getInputMethodListeners, getInputMethodRequests, getKeyListeners, getLocation, getLocation, getLocationOnScreen, getMouseListeners, getMouseMotionListeners, getMousePosition, getMouseWheelListeners, getName, getParent, getPeer, getPropertyChangeListeners, getPropertyChangeListeners, getSize, getSize, getToolkit, getTreeLock, getWidth, getX, getY, gotFocus, handleEvent, hasFocus, hide, imageUpdate, inside, isBackgroundSet, isCursorSet, isDisplayable, isDoubleBuffered, isEnabled, isFocusable, isFocusOwner, isFocusTraversable, isFontSet, isForegroundSet, isLightweight, isMaximumSizeSet, isMinimumSizeSet, isOpaque, isPreferredSizeSet, isShowing, isValid, isVisible, keyDown, keyUp, list, list, list, location, lostFocus, mouseDown, mouseDrag, mouseEnter, mouseExit, mouseMove, mouseUp, move, nextFocus, paintAll, postEvent, prepareImage, prepareImage, printAll, processComponentEvent, processFocusEvent, processHierarchyBoundsEvent, processHierarchyEvent, processInputMethodEvent, processKeyEvent, processMouseEvent, processMouseMotionEvent, processMouseWheelEvent, remove, removeComponentListener, removeFocusListener, removeHierarchyBoundsListener, removeHierarchyListener, removeInputMethodListener, removeKeyListener, removeMouseListener, removeMouseMotionListener, removeMouseWheelListener, removePropertyChangeListener, removePropertyChangeListener, repaint, repaint, repaint, requestFocus, requestFocus, requestFocusInWindow, requestFocusInWindow, reshape, setBackground, setBounds, setBounds, setComponentOrientation, setCursor, setDropTarget, setEnabled, setFocusable, setFocusTraversalKeysEnabled, setForeground, setIgnoreRepaint, setLocale, setLocation, setLocation, setMaximumSize, setMinimumSize, setName, setPreferredSize, setSize, setSize, setVisible, show, show, size, toString, transferFocus, transferFocusUpCycle

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

wordNet

public static InstanceOf wordNet

Holds a WordNet for approximate comparisons

array

public double[] array

Holds an array of precision/recall-values [p1,r1,p2,r2,...]

files

public static java.util.Set<java.lang.String> files

Holds the files for which ideal pairs exist

idealPairs

public static java.util.List<EvaluatePair> idealPairs

Holds ideal pairs

outputPairs

public static java.util.List<EvaluatePair> outputPairs

Holds output pairs

leilaPattern

public static final java.util.regex.Pattern leilaPattern

Pattern of LEILA output pairs

textToOntoPattern

public static final java.util.regex.Pattern textToOntoPattern

Pattern of textToOnto output pairs

text2OntoPattern

public static final java.util.regex.Pattern text2OntoPattern

Pattern of text2onto output pairs

cvPattern

public static final java.util.regex.Pattern cvPattern

Pattern of CV output pairs

cvSimplePattern

public static final java.util.regex.Pattern cvSimplePattern

Pattern of CV output pairs in their simple form

Constructor Detail

Evaluate

public Evaluate(double[] a)

Constructor from an array of precision/recall-values

Method Detail

paint

public void paint(java.awt.Graphics g)

Draws the precision/recall-dots

Overrides:

paint in class java.awt.Container

init

public void init()

Does nothing

Overrides:

init in class java.applet.Applet

addIdeal

public static void addIdeal(java.io.File f)
                     throws java.lang.Exception

Adds the ideal pairs from a given file. Ideal pairs are given by the form

 % Optional comments
 first word / second word
 first word / second word
 ...
 

Throws:

java.lang.Exception

addOutput

public static void addOutput(java.io.File f,
                             Evaluate.IESystem system)
                      throws java.lang.Exception

Adds the outputPairs pairs from a given result file. Skips index.lgo.

Throws:

java.lang.Exception

eval

public static double[] eval(int approx,
                            boolean sym)

Calculates precision and recall. Approx is the number of wordnet-hops allowed. May take long if approx!=0. sym says whether the relation is symmetric.

evalIdeal

public static double[] evalIdeal(int approx,
                                 boolean sym)

Calculates precision and recall according to the ideal metric. Approx is the number of wordnet-hops allowed. May take long if approx!=0. sym says whether the relation is symmetric.

main

public static void main(java.lang.String[] args)
                 throws java.lang.Exception

User interface

Throws:

java.lang.Exception