Thursday, April 13, 2017

QILT scores - NSW regional universities outscore Sydney G8

The Quality Indicators for Learning and Teaching (QILT) website provides prospective students with relevant and transparent information about Australian higher education institutions from the perspective of recent students and graduates.

One of the most interesting things about the data released this week is that it appears to confirm something that I have long suspected, that there is little if no correlation between between institutional prestige and university entrance scores and actual student experiences. 

The tables below compare aggregate rankings between NSW’s non-metro universities and the Universities of NSW and Sydney. I'm sorry that the tables are so messy.

Table One shows the percentage of students who rated their experiences positively against various indicators. While there is some variation in the answers to the various questions, the non-metros generally score better than the more prestigious Sydney institutions, with the University of New England ranking first. 
 
Table One: Student Experience - Undergraduate
Charles Sturt University Southern Cross University University of New England University of New South Wales University of Newcastle The University of Sydney National Average
Overall quality of educational experience

77.6%
(77.1% -78.2%)
8463responses
78.2%
(77.3% -79.0%)
3598responses
83.3%
(82.5% -84.0%)
3994responses
76.0%
(75.5% -76.5%)
14290responses
82.5%
(81.9% -83.0%)
8125responses
76.4%
(76.0% -76.9%)
14350responses
79.9%
Teaching quality

80.3%
(79.7% -80.8%)
8371responses
81.2%
(80.3% -82.0%)
3557responses
84.5%
(83.8% -85.2%)
3965responses
77.5%
(77.0% -77.9%)
14063responses
83.4%
(82.9% -84.0%)
8052responses
79.6%
(79.2% -80.0%)
14092responses
81.5%
Learner engagement

67.8%
(67.0% -68.7%)
4574responses
62.0%
(60.7% -63.2%)
2521responses
66.5%
(65.0% -68.0%)
1492responses
65.2%
(64.7% -65.7%)
14278responses
58.9%
(58.2% -59.6%)
8044responses
60.3%
(59.8% -60.8%)
14321responses
64.2%
Learning resources

84.6%
(83.9% -85.2%)
5809responses
84.0%
(83.1% -85.0%)
2774responses
87.6%
(86.5% -88.6%)
2201responses
83.0%
(82.6% -83.5%)
13327responses
88.2%
(87.8% -88.7%)
7716responses
81.0%
(80.6% -81.5%)
13487responses
85.2%
Student support

74.3%
(73.6% -75.0%)
7329responses
76.1%
(75.1% -77.1%)
3166responses
80.2%
(79.3% -81.1%)
3346responses
66.4%
(65.9% -67.0%)
11636responses
75.1%
(74.3% -75.8%)
6828responses
58.7%
(58.1% -59.3%)
11709responses
72.0%
Skills development

78.9%
(78.3% -79.4%)
8220responses
81.5%
(80.7% -82.4%)
3495responses
78.4%
(77.5% -79.2%)
3888responses
77.9%
(77.4% -78.4%)
13697responses
81.7%
(81.1% -82.3%)
7902responses
79.4%
(79.0% -79.9%)
13683responses
81.2%
\
Table Two looks at measures of graduate satisfaction. There is a little more variation here, although again the non-metros do a little better, with the University of New England a clear first.

Table Two: Graduate Satisfaction - Undergraduate
Charles Sturt University Southern Cross University University of New England University of New South Wales University of Newcastle The University of Sydney National Average
Overall satisfaction

79.7%
(78.7% -80.7%)
3081 responses
81.1%
(79.7% -82.5%)
1401responses
87.0%
(86.1% -87.9%)
1991responses
80.2%
(79.5% -80.8%)
6645responses
83.2%
(82.4% -83.9%)
4127responses
79.3%
(78.4% -80.1%)
4695responses
82.2%
Teaching scale

65.6%
(64.4% -66.8%)
3090responses
70.3%
(68.7% -71.9%)
1405responses
71.7%
(70.5% -72.9%)
1992responses
63.3%
(62.5% -64.0%)
6670responses
69.5%
(68.6% -70.4%)
4140responses
61.4%
(60.4% -62.4%)
4703responses
68.0%
Skills scale

79.1%
(78.1% -80.1%)
3088responses
82.5%
(81.2% -83.9%)
1404responses
85.2%
(84.2% -86.1%)
1992responses
82.1%
(81.5% -82.7%)
6666responses
87.6%
(87.0% -88.3%)
4136responses
80.9%
(80.1% -81.7%)
4698responses
84.1%

Table Three looks at graduate employment. The results are interesting, but need to be interpreted with some care.

There is a considerable range in the proportion of graduates who go onto further full time postgraduate study from just 6.1% at Charles Sturt to 29.9% at the University of Sydney. Excluding these two as outriders, the percentages range from 14.7% at UNE to 17.9% at UNSW.

The figure for full time employment is the % of graduates available for full time work who were in full time work four months after graduation. The median salary figure is the median for those graduates in full time employment.  Charles Sturt had the best full time employment record followed by UNE and then UNSW.

The overall employment number includes those in full time employment plus casual and temporary. Some of the second appear to be also included in the full time study category.
Table Three: Graduate Employment - Undergraduate
Charles Sturt University Southern Cross University University of New England University of New South Wales University of Newcastle The University of Sydney National Average
Full-time employment

83.9%
(83.1% -84.7%)
4031responses
68.0%
(66.2% -69.8%)
1319responses
77.3%
(76.0% -78.6%)
1871responses
76.4%
(75.6% -77.2%)
5381responses
68.4%
(67.4% -69.3%)
4431responses
70.4%
(69.3% -71.4%)
4171responses
69.5%
Overall employment

93.9%
(93.4% -94.4%)
4721responses
86.9%
(85.9% -88.0%)
1724responses
88.7%
(87.8% -89.5%)
2293responses
89.0%
(88.5% -89.5%)
6151responses
90.5%
(90.0% -91.0%)
5226responses
87.3%
(86.7% -88.0%)
5285responses
88.6%
Full-time study

6.1%
(5.7% - 6.6%)
4944responses
16.4%
(15.4% -17.4%)
2069responses
14.7%
(13.8% -15.5%)
2596responses
17.9%
(17.3% -18.4%)
7419responses
17.1%
(16.5% -17.6%)
6066responses
29.9%
(29.2% -30.6%)
7032responses
21.6%
Median salary

$60,000
($59,900 -$60,100)
2850responses
$57,000
($55,400 -$58,600)
759 responses
$60,000
($59,300 -$60,700)
1217responses
$60,000
($59,400 -$60,600)
3321responses
$57,000
($56,400 -$57,600)
2546responses
$56,000
($55,200 -$56,800)
2444responses
$56,000

I have yet to dig into the detail at subject level where the pattern is more varied. Still, the apparent absence of any correlation between between institutional prestige and university entrance scores and actual student experiences remains interesting.

5 comments:

Winton Bates said...

Table 2 is particularly interesting. Satisfaction after graduation seems to me to be worth having.

marcellous said...

I've never found these league tables very easy to interpret. Whenever I've known something about them, I've seen about a million holes in them. For example, when I went through law school, if you wanted to be a solicitor, you had to go to College of Law for half a year after that. If you were lucky you might get into the first course; if you weren't, you waited for the second. Some people got work in the intervening period; some already had full-time employment; others went off for a big trip around the world. Can you imagine what sense the graduate employment statistics made of that?

Jim Belshaw said...

I agree with you, marcellous. After all, I have been complaining about the use and abuse of league tables for a long while!

I do think that the satisfaction measures and especially Table Two as noted by Winton do provide a useful measure. However, I note that one of my twitter colleagues commented that perhaps the satisfaction levels for regional students was higher because they started with lower expectations! Or, conversely, that of city students lower because they started with higher expectations.

Given the prevalent use of league tables in marketing that so feature the metros because of the way they are structured, something that distorts the marketplace, the new numbers as "official" numbers are quite a useful corrective.

Anonymous said...

This should be hugely reassuring for those vitterinary wallahs from Canberra - surely?

kvd

Jim Belshaw said...

Perhaps not for those regulatory scientists who have degrees and gone to Canberra in the first place and who do not want to move, kvd. But perhaps for future students in regulatory science who go to UNE. It's interesting. APVMA seems to have been struggling for a while to get staff. There doesn't appear to be any university level training in Canberra in the field, so have persuade people to go there May well be a better position when UNE is graduating.