Methodology

Scoring: Rasch model with an EAP estimate

We use the Rasch model, the one-parameter model of item response theory. The probability that a person with ability θ (theta) answers an item of difficulty b correctly is:

P(correct | θ, b) = 1 / (1 + e^-(θ - b))

• Estimate. θ is the expected a posteriori (EAP) value: the mean of the posterior over a grid from -4 to +4 in steps of 0.01, with a standard normal prior. EAP gives a finite estimate even when every answer is right or every answer is wrong.
• Uncertainty. The standard error is the posterior standard deviation. The 95% range is θ plus or minus 1.96 standard errors.
• Scale. Estimated IQ = 100 + 15θ, rounded and kept between 65 and 160. This puts the result on the same deviation scale as modern clinical tests (mean 100, SD 15) (Wikipedia: Intelligence quotient).
• Skips. A skipped or unanswered item counts as wrong. Time does not change the score. The test submits automatically at 20 minutes.
• Five-area breakdown. Each section is scored on its own items with the same method and shown as a band from 1 to 10. Sections have 3 to 9 items, so these bands are much less precise than the overall estimate.

Raw score to estimated IQ

Because the Rasch estimate depends only on the number of correct answers, the whole scoring rule fits in one table. These values are computed from the scorer with the current difficulty values.

At the middle of the scale (13 correct) the 95% range runs from 92 to 116. For comparison, clinical full-scale IQ scores from long, supervised batteries are usually reported with a range of around plus or minus 5 points at 95% (Wechsler, 2008). A 25-item online test is less precise, and the table shows by how much.

Floor and ceiling

A perfect score gives 141, and no correct answers gives 65. The test cannot tell apart people above 141: it does not have enough hard items. This is a ceiling effect, and we state it rather than stretch the scale. Anyone who scores at the ceiling should read the result as “141 or higher”.

Calibration plan

The difficulty values are design priors: each template's parameters were built to hit a target difficulty, but they have not yet been fitted to human response data. The engine stores every response with the item's template, parameters and seed so the values can be re-estimated.

1. Collect responses. Once a template and difficulty level has at least 300 responses, fit its difficulty by marginal maximum likelihood.
2. Write the fitted values to the calibration file the generator reads. Items without enough data keep their design prior.
3. Publish the change on this page with the date, and rerun the AI benchmark so the comparison stays like for like.

Until that first fit runs, the raw score table above is the scoring rule.

Limitations

• No norming sample yet. The scale is anchored by design difficulties, not by a representative sample of adults. We do not claim that our 100 equals a clinical 100.
• No validity study. We have not compared IQ Octopus scores with clinical tests and quote no correlation or accuracy figure.
• No age norms. Clinical IQ compares you with people your own age. This test does not ask your age and is designed for adults.
• Short test. 25 items give a wide range and a ceiling of 141.
• Unsupervised conditions. Device, distractions and fatigue are outside our control.
• Practice effects. Retaking cognitive tests raises scores by about a quarter of a standard deviation on average (Hausknecht et al., 2007), even with new items.

Questions about the method, or an error to report? Email support@iqoctopus.com. Our editorial standards explain how we handle corrections.