Object

com.enriquegrodrigo.spark.crowd.methods

CATD

Related Doc: package methods

Permalink

object CATD

Provides functions for transforming an annotation dataset into a standard label dataset using the CATD algorithm.

This algorithm only works with continuous label datasets of type types.RealAnnotation:

The algorithm returns a CATD.CATDModel, with information about the class true label estimation and the annotators weight

Example: 

  1. import com.enriquegrodrigo.spark.crowd.methods.CATD
import com.enriquegrodrigo.spark.crowd.types._
sc.setCheckpointDir("checkpoint")
val annFile = "data/real-ann.parquet"
val annData = spark.read.parquet(annFile)
//Applying the learning algorithm
val mode = CATD(annData.as[RealAnnotation])
//Get MulticlassLabel with the class predictions
val pred = mode.getMu()
//Annotator weights
val annweights = mode.getAnnotatorWeights()
Version

0.2.0

See also

Q. Li, Y. Li, J. Gao, L. Su, B. Zhao, M. Demirbas, W. Fan, and J. Han. A confidence-aware approach for truth discovery on long-tail data. PVLDB, 8(4):425–436, 2014.

Linear Supertypes
AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. CATD
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Type Members

  1. class CATDModel extends AnyRef

    Permalink

    Model returned by the CATD algorithm

    Model returned by the CATD algorithm

    Version

    0.2.0

Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any

  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any

  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any

  4. def apply(dataset: Dataset[RealAnnotation], iterations: Int = 5, threshold: Double = 0.1, alpha: Double = 0.05): CATDModel

    Permalink

    Applies the CATD learning algorithm.

    Applies the CATD learning algorithm.

    dataset

    The dataset over which the algorithm will execute (types.RealAnnotation

    iterations

    Maximum number of iterations of the algorithm

    threshold

    Minimum change in MSE needed for continuing with the execution

    alpha

    Chi-square alpha value for the weight calculation

    returns

    CATD.CATDModel

    Version

    0.2.0

  5. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any

  6. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  7. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef

  8. def equals(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any

  9. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )

  10. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any

  11. def hashCode(): Int

    Permalink
    Definition Classes
    AnyRef → Any

  12. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any

  13. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef

  14. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef

  15. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef

  16. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef

  17. def toString(): String

    Permalink
    Definition Classes
    AnyRef → Any

  18. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  19. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  20. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Inherited from AnyRef

Inherited from Any

Ungrouped