Giter Club home page Giter Club logo

metasdt's Introduction

metaSDT

R-CMD-check DOI R-CMD-check

This is an R implementation of Maniscalco and Lau’s methods of calculating metacognitive SDT measures using maximum likelihood estimation and minimization of the sum of squared errors.

Installation

You can install metaSDT’s development version from GitHub with:

# install.packages("devtools")
devtools::install_github("craddm/metaSDT")

For further details on metacognitive/Type 2 SDT, see their website at http://www.columbia.edu/~bsm2105/type2sdt/ and the following publications:

Maniscalco, B., & Lau, H. (2012). A signal detection theoretic approach for estimating metacognitive sensitivity from confidence ratings. Consciousness and Cognition, 21(1), 422–430. doi:10.1016/j.concog.2011.09.021

Maniscalco, B., & Lau, H. (2014). Signal detection theory analysis of type 1 and type 2 data: meta-d’, response-specific meta-d’, and the unequal variance SDT mode. In S. M. Fleming & C. D. Frith (Eds.), The Cognitive Neuroscience of Metacognition (pp.25-66). Springer.

If you use these functions, cite the above papers and scripts on which it is based.

The fit_meta_d_SSE and fit_meta_d_MLE commands require data in the same format outlined on M & L’s webpage, as follows:

Suppose there are two stimuli, A, and B, and three confidence ratings - high (3), medium(2), and low(1). The possible responses are

A3, A2, A1, B1, B2, B3.

Input to the function should be counts for each of these responses separately for each stimulus type.

So for example:

library(metaSDT)
nR_S1 <- c(100, 50, 30, 20, 10, 4)
nR_S2 <- c(4, 20, 21, 35, 60, 90)
fit_MLE <- fit_meta_d_MLE(nR_S1,
                          nR_S2)
fit_MLE
#>         da s meta_da     M_diff  M_ratio    meta_ca   t2ca_rS1  t2ca_rS2
#> 1 1.845043 1 1.87284 0.02779706 1.015066 0.07108153 -1.0297441 0.5524547
#> 2 1.845043 1 1.87284 0.02779706 1.015066 0.07108153 -0.3710477 1.2052824
#>        logL est_HR2_rS1 est_HR2_rS2 est_FAR2_rS1 est_FAR2_rS2 obs_HR2_rS1
#> 1 -444.1742   0.8469277   0.8052557    0.4938669    0.4352026   0.8328717
#> 2 -444.1742   0.5489194   0.4885083    0.1273860    0.1027034   0.5549400
#>   obs_HR2_rS2 obs_FAR2_rS1 obs_FAR2_rS2       t1c1
#> 1   0.8104223   0.53479853    0.4154589 0.07002653
#> 2   0.4860737   0.09157509    0.1207729 0.07002653
fit_SSE <- fit_meta_d_SSE(nR_S1,
                          nR_S2)
fit_SSE
#>         da meta_da     M_diff  M_ratio    meta_ca s t2ca_rS1 t2ca_rS2
#> 1 1.845043    1.89 0.04495688 1.024366 0.07173282 1   -1.057    0.568
#> 2 1.845043    1.89 0.04495688 1.024366 0.07173282 1   -1.057    0.568
#>          SSE est_HR2_rS1 obs_HR2_rS1 est_HR2_rS2 obs_HR2_rS2 est_FAR2_rS1
#> 1 0.00246936   0.5386789   0.5549400   0.7999733   0.8104223     0.118341
#> 2 0.00246936   0.5386789   0.8328717   0.7999733   0.4860737     0.118341
#>   obs_FAR2_rS1 est_FAR2_rS2 obs_FAR2_rS2       t1c1
#> 1   0.09157509    0.4214078    0.4154589 0.07002653
#> 2   0.53479853    0.4214078    0.1207729 0.07002653

Output is a data frame with m-ratio etc.

metasdt's People

Contributors

craddm avatar

Stargazers

 avatar Francisco Garre Frutos avatar  avatar Hu Chuan-Peng avatar Ladislas Nalborczyk avatar Mohammad Dastgheib avatar  avatar Micah Galen Allen avatar Linda avatar

Watchers

James Cloos avatar  avatar  avatar

metasdt's Issues

Inconsistency in Data Format Instructions between README and R Script

It appears there is a discrepancy between the data format instructions provided in the README file and the format used in the R script.

Issue Description:
In the README file, the suggested format for preparing data for the function is listed as:

A1, A2, A3, B3, B2, B1.

However, upon examining the R script (particularly in this section), I noticed that the data format is actually:

A3, A2, A1, B1, B2, B3.

Impact:
This inconsistency in data format instructions could lead to incorrect implementation of the function, ultimately affecting the resulting metacognitive SDT measures.

Suggested Resolution:
Could you please clarify which format is the correct one? Additionally, updating either the README or the R script to ensure consistency would be greatly beneficial for future users.

Thank you for your attention to this matter. I appreciate the work you are doing with this library and look forward to your response.

Allow specification of stimulus levels

Currently the functions assume that the first level of whatever factor is inputted as the "stimulus" factor is the "signal", which is not always correct. Would be better if the stimulus level of the factor could be specified.

fit_meta_d_SSE producing very large meta_da values

Hi there,

When comparing results from fit_meta_d_SSE and fit_meta_d_MLE I sometimes get very different results, with very large meta_da values from fit_meta_d_SSE. For example, the below example will produce a meta_da value of 0.932558 from fit_meta_d_MLE but a value of 5 from fit_meta_d_SSE

nR_S1 <- c(0,  2, 16,  7,  0,  0,  0,  0, 10, 15,  4,  0)
nR_S2 <- c(0,  0,  0,  3,  1,  0,  0,  0, 10, 22, 18,  0)


metaSDT::fit_meta_d_SSE(nr_s1 = nR_S1, nr_s2 = nR_S2)
metaSDT::fit_meta_d_MLE(nr_s1 = nR_S1, nr_s2 = nR_S2)

General package improvements required (AKA TODO list)

The package really needs some love.

  1. Much better documentation
  2. Implement regression approaches to Type I SDT measures
  3. Investigate regression approaches to Meta-d
  4. More consistent output across the different methods
  5. Add some plotting functions?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.