Receiver Operating Characteristic (ROC) suite

Brief description

The Receiver Operating Characteristic (ROC) evaluates map similarity considering a reference binary pattern. ROC compares the amount of true positive and false positive cells through an incremental binary classification (Mas et al. 2013a). ROC is commonly used in GIS to evaluate spatial predictions versus observed data. An ROC suite developed by Mas et al (2013) is available on Dinamica-EGO. There are some study cases that apply ROC to evaluate binary predictions through a probability or suitability map using a suite of functors and submodels on Dinamica-EGO.

About metrics

The main ROC metrics used to evaluate the results are the area under curve (AUC) and the partial area under curve (pAUC). Fig. 1 presents the standard ROC graph of true positive and false positive. The red diagonal curve represents a hypothetical model that predicts the same number of hits and false alarms. The suitability maps are interpreted on the ROC as prediction curves compared with the fixed diagonal. Each suitability evaluation generates a new curve on the graph. Curves over the fixed diagonal represent models that perform better than a random model. The final gain (relative to the suitability map) is summarized by the AUC measure. An equivalent metric can be applied to measure hit rate or error rate, this partial measure is called pAUC, as illustrated in the figure 01.

Fig. 1 – ROC curve and respective metrics.

Cases

References

Resources

The ROC suite models, submodels and example data are in guidebook dinamica\models\roc suite, part of guidebook_dinamica.7z. The folder also contains the installation instructions and application examples by Mas (2013b), in Spanish (analisis roc instrucciones.pdf).

☞Next: Final Remarks


This documentation was generated with the help of large language models (LLMs) and may contain errors. Please verify critical information.