<% ' expirt.asp ' sc 0 - percentile ' 1 - z ' 2 SAT scale ' 3 - no image '~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ function rsetit (k) k1=trim(cstr(k)) while len(k1)<3 k1="0" & k1 wend rsetit=k1 end function '~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ function afmt (x,d) ' presents x numbers to d decimal places ' response.write x & " " & d &"
" y=CINT((10^d)*x )/10^d y=cstr(y) if d>0 then if instr(y,".")=0 then y=y&"." while len(y)-instr(y,".") 2.8 then wh="high" th1=th select case sc case 0 ' percentile aa= "percentile" bot="0" top="100" case 1 ' z aa="z-score" bot="-3.0" top="+3.0" case 2 ' sat aa="SAT score" bot="200" top="800" end select 'if cth>-2.8 and cth< 3.0 and csee<2.0 then low=csng(th)-csng(see) hi=Csng(th)+csng(see) if low < -3.0 then low=-2.99 if hi > 3.0 then hi=2.99 select case sc case 0 low=100*prbz(low) hi=100*prbz(hi) low=afmt(low,0) hi=afmt(hi,0) Case 1 low=afmt(low,2) hi=afmt(hi,2) case 2 low = 500+100*low hi=500+100*hi low=afmt(low,0) hi=afmt(hi,0) end select 'end if if cpr<.50000 then ab="wrong" sn="<" else ab="right" sn=">" end if %>

Explanation of the Ability Estimation graph

This graph lets us see how well CAT is doing.

The goal of CAT, like any other test, is to obtain an accurate estimate of your true score (a <%=aa%> score of <%=tr%>; shown in red). The current estimate is a <%=aa%> score of <%=cth%> (shown with a black line).

<% if th>-2.8 and th< 3.0 and see<2.0 then%> The yellow band shows the standard error of the estimate of your ability. While you know your true score (you specified a true score at the start of this tutorial), there is no way for the computer or anyone to actually know the value of this parameter. All we can do is estimate its value based on your item responses and information we have about the items you have taken. The standard error helps us evaluate the precision of our current estimate. A small standard error indicates that the current estimate appears to be fairly accurate. If you repeated this testing a large number of times, we would expect your scores to vary plus or minus one standard error 68% of the time. With the current standard error of <%=csee%> z-score units, the confidence interval is <%=low%> to <%=hi%>. <% if sc=0 then%> (The standard error is not symmetrical around the estimated ability, because you are using percentiles. It would be if you were using z-scores or SAT scores.) <% end if %> <% else %> This graph usually shows the standard error associates with the current estimate of ability. The standard error is not shown, however, because <% if see>2.0 then %> it is currently very wide and would cover most of the graph. <% else %> your ability estimate is so <%=wh%>. <% end if %> <% end if %> As more items are administered, the standard error, and the size of the confidence interval, will decrease.

<% if sc>0 then %> The familiar bell shaped curve on the graph is the relative number of people in the population at each <%=aa%> score level. <% else %> The upside down semi-circle shows the relative number of people in the population at each percentile score. Had the metric been z-scores or SAT scores, the semi-circle would have appeared as the familiar bell-curve or "normal" curve. <% end if %>

What to look for:

Notice the width of the confidence interval. If it is wide or not shown, then the estimate of ability is not very precise.

Notice that the width of the confidence interval gets smaller as the number of items administered gets larger.

Notice that the estimate of your true ability gets closer to your true ability as more items are given.

Try clicking your back button and changing the score before re-submitting your answer. Notice the difference in the size of the confidence interval depending on whether you are using percentiles or z-scores. A few items can make a big difference in the middle of the percentile score distribution. In the middle of the ability distribution, standard errors in terms of percentiles are quite large. At the tails, the opposite is true. A few items do not make a big difference. Standard errors tend to be small at the tails.

From: Rudner, Lawrence M. (1998). An On-line, Interactive, Computer Adaptive Testing Mini-Tutorial, http://ericae.net/scripts/cat