The PRE is a Bad Test. Here’s Why.

For the past six months, I have been studying what I have facetiously named the PREblem: the high failure rate of preservice teachers in Michigan taking the Pearson Professional Readiness Exam. In October 2013, this test replaced the Basic Skills Test, and as a consequence of its allegedly more rigorous content and its higher cut scores, teacher candidates in every discipline and at every Michigan teacher preparation institution are failing in huge numbers. How huge? Some recent pass rates (courtesy of the Center for Michigan’s Bridge, January 2015):

U-M Ann Arbor, 71%
Michigan Tech University,55%
Hope College, 43%
Calvin College, 42%
Grand Valley State, 41%
Michigan State University, 41%
Lake Superior State, 39%
Concordia University, 37%
Aquinas College, 36%
Cornerstone University, 35%
Madonna University, 32%
Oakland University, 29%
Spring Arbor University, 28%
Saginaw Valley State, 27%
Eastern Michigan University,23%
Northern Michigan University,23%
Adrian College, 22%
Central Michigan University, 22%
Andrews University, 20%
Western Michigan University,20%
U-M Dearborn, 19%
Wayne State University, 19%
Baker College, 18%
U-M Flint, 18%
Ferris State University, 17%
Alma College, 16%

Pearson would likely argue that these failure rates prove the rigor of the test. After all, as one Pearson representative told me at a PRE standards-setting workshop , “You have to draw the line somewhere,” when it comes to allowing aspiring teachers into the profession. If only 19 percent of Wayne State students can manage to pass the PRE, then only 19 percent of them should be allowed to teach, right?

Well, no. We cannot allow a corporate-designed multiple-choice test to act as a gatekeeper into the teaching profession. This is more than a minor issue. It is a full-blown crisis in Michigan. But how to begin critiquing a standardized test that seems to have a stranglehold on our teacher preparation institutions? One way is to question its validity. There are dozens of ways of measuring test validity (see the methods advocated by Pearson competitor College Board), and while I am not a pyschometrician, it is clear that the PRE violates at least two key principles of test validity.

The first is called consequential validity. A definition the College Board site:

Some testing experts use consequential validity to refer to the social consequences of using a particular test for a particular purpose. The use of a test is said to have consequential validity to the extent that society benefits from that use of the test. Other testing experts believe that the social consequences of using a test—however important they may be—are not properly part of the concept of validity.

Messick (1988) makes the point that “. . . it is not that adverse social consequences of test use render the use invalid but, rather, that adverse social consequences should not be attributable to any source of test invalidity such as construct–irrelevant variance.”

What are the adverse societal consequences of the Professional Readiness Exam? First, in colleges of education across Michigan, aspiring teachers are in limbo. The problem began with large numbers of students failing the new PRE rolled in 2013; this led many colleges of education to temporarily waive the previous admission requirement of the passing entrance-level exam. Students with failing scores entered their programs, completed their their first field placement (typically teacher assisting for a semester), and have even passed their subject area tests (also sold by Pearson). But they are still unable to pass the PRE and are thus prevented from student teaching. My university currently has 60 students caught in limbo, and other schools are reporting similar numbers. If Michigan does indeed have a teacher shortage (especially early childhood, world languages and special education), then holding back potential teachers will only exacerbate it.

Teacher candidates are also require to pay Pearson for each test taken–sometimes, up to 5 or 6 times, at $50.00 apiece. The Michigan Department of Education does kindly recommend that that students who have failed four or more times ” seek academic counseling from college/university staff in an attempt to overcome testing deficiencies.” Of course, Pearson does provide a practice exam for only $29.00. So, is all of this money spent a significant social consequence? Yes–considering that the money might otherwise go toward tuition, loan repayment, or books.

Moreover, the PRE is preventing teachers of color and teachers for whom English is a second language from entering the job market. In a letter to State Superintendent Brian Whiston, The Michigan Association for Colleges of Teacher Education points out that African-American and Hispanic teacher candidates have substantially lower PRE pass rates than white candidates; this while more students in Michigan speak Spanish as a first language; this while city schools in Grand Rapids, Flint, and Detroit are facing enormous issues in attendance, graduation rates, teacher turnover, and more.

A second and equally important measure of test validity is content validity. Content validity measures whether the body of knowledge that the test includes reflects the subject area that the test is meant to measure. In the case of teaching, Pearson has determined that all teachers must know three basic subject areas: reading, writing, and math. There is a kind of back-to-basics appeal to this curricular trio–reading, writing, and arithmetic. Here’s the problem: I taught high school English for eight years, from 1994 to 2001. I have an MA in literary studies from Michigan State, and a Ph.D. in English (specialization English Ed) from Western Michigan. For the last 12 years I taught methods courses and have supervised teacher assistants and student teachers in dozens of secondary schools in west and mid-Michigan. I also publish books and articles on teaching English.

In short, I am qualified to teach secondary English. But when I retook the PRE this fall, I failed the math portion. I haven’t done Algebra or Trigonometry in over 20 years, and while my math memory may fail me, I am pretty sure I never had to teach trigonometry in my British literature classes. I did have to weight and average grades, and I may have drawn a triangle or two on the board, but that is about all the math I did, lucky for me.

So, why should all teachers need to master difficult math concepts–ones they will never, ever use in their careers? I would much rather see all teachers certified in the fields of neurodiversity, gender awareness, English language learning, and other more applicable areas. I’ve got nothing against algebra or tough standards, but the content of the entry-level test simply does not align with the actual teaching field.

Even the skills that all teachers do need to possess–the ability to read and to write–are represented problematically on the PRE. A third kind of validity, construct validity, means that a test measures what it is supposed to measure; a valid test asks you to perform the same kinds of skills that the actual subject matter asks you to perform. This works in the case of math (my nemesis), since the kinds of problems the PRE includes represent the kinds of problems actual mathematicians do, if we assume that they do so with a time limit and with only a small whiteboard and dry erase marker (sorry, no paper or pencils allowed).

With the writing test in particular, however, there is a serious mismatch between what writers actually do and how the test frames what writers do. Given this mismatch, it is not a surprise that the writing scores on the PRE are consistently the lowest; indeed, writing scores on standardized tests as a whole are always the lowest. There is just no way to cram what writers do into a limited-time, artificial exercise. The best way to illustrate this idea is with a sample question from Pearson’s PRE study guide:

Given a short passage, the test taker is asked: “Which of the following parts should be edited to correct an error in subject-verb agreeement?”

Setting aside the ridiculous nomenclature (no writer has ever called a sentence a part), and the problematic idea that editing writing involves discriminating between three error-free sentences and one incorrect sentence (imagine a copyeditor saying to herself, “I know one of these sentences has an error in subject-verb agreement. If I only knew which one!”), we are still left with the implausible,even absurd scenario that the creation of this question involved: presumably, the sentence was once correct and was made incorrect for the purpose of the test, so that the original sentence was

This pioneering, volunteer-based approach that she developed to bring eye-care services to underserved populations has had a positive effect on the lives of countless people.

This sentence was then changed to the incorrect version (with an effect/affect error thrown in for good measure):

This pioneering, volunteer-based approach that she developed to bring eye-care services to underserved populations have had a positive affect on the lives of countless people.

In a process unlike anything that writers actually do, the test taker is supposed to identify this manipulation and return the part to its original, correct stage. It reminds me of a children’s television program or coloring book: Swiper (or similar villain) has taken all the berries from the secret forest! Can you find them and return them to the berry bushes?

And on it goes for 42 multiple choice questions. Then, the test taker gets to compose two constructed responses–one analytical argument; and one expository. Again, there is little to no similarity between what the test taker is asked to do and what real writers actually do. There is no writer in the world who is asked to write about a topic he or she is not interested in; to follow a basic organizational pattern that can be easily assessed; to write without the chance for revision; and do to so within a limiting time frame.

All this is to say that teacher educators (like me) would be better off channeling our energy into fighting this test instead of scrambling to prepare our students to do better on it. In the short term, we can offer all the support possible (workshops, one-credit classes, web resources), but our long-term goal should be to kick this test to the curb. It is deeply flawed, and it is hurting our students and our state.

Download PDF
Posted in News | Tagged , | Comments Off on The PRE is a Bad Test. Here’s Why.

Twitter On Its Last Legs?

Ignoring for the moment that blogging, too, is on its last legs, and that this blog is living (or dying) proof of that demise, it is looking more and more like Twitter is being abandoned. The Atlantic ran this eulogy for the micro-blogging platform this past summer, and other bloggers (like Bad Words) have made similar pronouncements more recently. If this is the case–and the activity on Twitter does seem to be on the decline–then perhaps my 2010 post calling the service a “flash in the pan” was right, at least in the long term.

There are likely a few reasons people are leaving the service, but I think the main issue with Twitter was always its usefulness: unlike Facebook, which people user for a wide range of personal and professional reasons, Twitter never offered a genuine purpose, except perhaps in fleeting moments when it seemed to scoop the mainstream and online media, as in the Arab Spring. Twitter serves well in these contexts, when the news is censored, too slow, or too top-down to give the real story. But these kind of events seem to be few and far between: who can remember if Twitter played a role in the Baltimore riots, for example?

Instead, what Twitter offers is an endless stream of insider chatter, a kind of echo chamber where various groups fall into predictable patterns of approval and rejection. It does not really matter what Donald Trump tweets to his followers on Twitter, or that he and Jeb Bush exchange indignant, 140-word insults about 9/11. Each partisan set of devotees retweet the remarks, breathlessly, as if they were somehow compelling or urgent. But no one cares.

It may be the limits of the form–the cap of 140 characters seems to breed invective and ignorance–but more importantly, it is just not as relevant or meaningful as Facebook seems to be. For better or worse, Facebook has integrated itself into our lives, becoming in a short time the place where we exchange family photographs, pass along news stories, watch cat videos, grieve, and celebrate. Twitter never wove itself into our social fabric, and its time has apparently come.

Download PDF
Posted in News, Web 2.0 | Tagged | Comments Off on Twitter On Its Last Legs?

Digital Humanities Summer Institute

This week, I am attending the Digital Humanities Summer Institute at the University of Victoria. It has been a terrific experience so far, mostly because it is forcing me to think about digital technologies in a new way. I’ve come at digital technology from a pedagogical standpoint for so many years, but the Digital Humanities field is something completely different, as my week-long course in electronic literature is proving. I’m listing some resources here in an attempt to organize my thinking:

Digital Humanities Resources

Electronic Literature Organizations and Databases

Composing Tools

Examples of Electronic Literature, Multimodal Texts, and Alternate Reality Works

Books and Journals

Download PDF
Posted in News | Tagged | 11 Comments

Where Have You Gone, WordPress?

I have advocated for WordPress over and over again since beginning this blog in 2005 (although back then, I used a now-defunct engine called Blogspirit to host this site). WordPress is, or maybe was, the best thing on the market for a highly customizable, easy-to-use web site or blog. I’ve made dozens of WordPress hosted sites over the years for my English courses (yes, I am still a Blackboard holdout–they’ll never take me alive), and I’ve championed it with pre-service teachers as well. I’m not alone: something like 25 percent of the most visited sites are run on the WordPress platform. I should probably get a WordPress T-shirt.

But something happened at WordPress.com. Likely they got too big, or like Ning, decided that giving the good stuff away for free wasn’t good business sense. In any case, there are a number of things that you USED TO be able to do with WordPress.com, but can no longer do. Here is my list of gripes:

EmbeddingEmbedding anything (a Scribd document, a Google map, a Vimeo video) used to be oh-so-simple, even on the free WordPress.com sites. You just cut and pasted the embed or even i-frame code into your post. And it worked. But now, the only real embed that WordPress allows is from Youtube/Google, which is very limiting. Even with the Premium upgrade, it is still nearly impossible to pull off. Here’s a recent conversation I had about this subject with a very nice WordPress support person:

rozemar I want to embed a slideshow in a Premium account blog. The Gallery slideshow does not look good, and I can’t seem to modify the CSS sufficiently to make it look the way I want it to. Is there a way to enable the embed or iframe HTML tag in a Premium account?
DW hi there
DW for security reasons, certain types of external code can not be used on WordPress.com sites, and iframe is one of them
DW you can read more about that here: http://en.support.wordpress.com/code/
DW what’s the issue you’re having with the slideshow? maybe i can help with a suggestion
rozemar It’s frustrating to pay the Premium amount and to be so limited in the kinds of things I can do.
rozemar The slideshow has the wrong color background (black), is too large, and can’t be moved around on the page very well.
DW understandable. our service is managed, which is great for those who don’t want to be responsible for adding extra features, but not as exciting for those who do 😉
DW would one of the other gallery styles work for you instead of the slideshow?
DW i agree with you, the slideshow layout isn’t one of my favourites. i’m a big fan of the tiled gallery layout, or thumbnails
rozemar I want a simple, autoplaying slide show with no background and no controls. Other gallery styles show thumbnails. I just want a single image replaced by another, by another, etc.
DW got it. are you familiar with our css forum?
DW our CSS support is done in the CSS Customization forum by expert staff and volunteers: http://en.forums.wordpress.com/forum/css-customization
rozemar No. I did find some CSS versions online, but they are not working well.
DW that’s where you’ll find the most expert help for WordPress.com
rozemar I was hoping that an individual site could have the embed or iframe tags enabled.
DW no, all sites on the network are subject to the same rules. for certain services, we build special embed functionality
rozemar If I remember correctly, even the free version of wordpress.com used to have embed and iframe tags.
DW you can use whatever code you like on self-hosted WordPress.org installations. here’s a bit of info on the difference: http://en.support.wordpress.com/com-vs-org/
rozemar Yes, I have a self-hosted WordPress at secondaryworlds.com.
DW yes! exactly.
rozemar So, what does it take to get the special embed functionality?
DW it depends on what you want to embed. we develop special embed features for certain services, based on demand
DW for instance, you can embed a youtube video just by copying the url of the video into your post, eliminating the need for iframes or anything else
DW we have the same functionality for flickr, but I’m trying to figure out if that extends to flickr galleries or just single images
rozemar Okay, so a Flickr gallery might work?
DW I’m not sure, I’m trying to find out right now :)
rozemar The problem with Youtube is that it doesn’t autoplay in WP, which is what I want.
DW of course
DW okay, did a test and it doesn’t look like it can display a flickr album
rozemar Boo.
DW just individual photos :(
rozemar So, I’m creating a site for a client with a puppy breeding home business. Site is cpuppies.wordpress.com
DW yeah, sorry to be the bearer of bad news. for slideshows, you’ll need to work with the current slideshow for WordPress.com
DW sure
rozemar Or try to modify the CSS, which may not respond well to mobile access.
DW yes, but I do really recommend checking with the css forum. they’ve seen these same modification requests time and time again, I’m sure
rozemar Hard to tell client that $99.00 year can’t get a decent slideshow :(
DW sure, understandable
rozemar Okay. Thanks for you help.
DW I think a lot of the slideshow design focus goes to the full-screen slideshow that’s triggered from clicking a gallery item
DW the Carousel is really popular, but you need to display your images in another type of gallery to access it
DW you’re welcome! wish I had a better workaround for you on the spot. but best of luck with the CSS, that might make a difference for you

So, the service is now managed, and embeds from sources constitute a security threat. Hmm. Makes me wonder why they didn’t represent a threat before . . .

I’m also a little miffed about the lack of access to the PHP and HTML of the site with a premium account. Paying $99.00 per year should let me modify these templates, but again, WordPress seems to be on lockdown. If I want to modify a theme to allow categories to be shown with each post, for instance, it is impossible, except on a self-hosted installation like this one.

The 3GB limit is also a little restrictive. 3 GBs is really, really tiny if you plan to include a lot of videos in your media library. Most cloud storage starts with at least 5GB (though Dropbox is down to 2GB, it appears).

A final gripe is about the wonkiness of the responsive design layout, particularly in adding images to a post. I can’t get the images to not wrap unexpectedly, for instance, and while every monitor/device looks different, it would be great to have an interface that makes responsive design a little bit simpler.

The newish login process is also needlessly confusing. I go to WordPress.com, but from there I am taken to a blue screen with some stats (which I never check), and I then have to go to WP Admin, at which time I may or may not be on the correct blog. It’s also almost impossible to find where to create a new blog! Seriously, I had to leave the site and Google it to figure it out.

Okay–that’s my big gripe. Of course, I am still using it in my courses. Still maybe the best game in town, but not as good a game as it used to be.

Download PDF
Posted in Cloud Computing, News | Tagged | 18 Comments

The Global Reach of the LAJM

The LAJM is published through Scholarworks, a service that our university uses to encourage academic publishing. It is a very powerful tool that archives and indexes all of the past issues of the journal, while providing editorial tools to manage the production of the journal. One of the coolest things that Scholarworks does, in my opinion, is to keep track of downloads. As the map below shows, downloads of LAJM articles occur all over the world, and more often than I would think.


Download PDF
Posted in News | Tagged | Comments Off on The Global Reach of the LAJM

Pearson and Data: It’s a One-Way Street

Call me curious. On January 17, I took the Michigan Test for Teacher Certification (MTTC) subject area test in English. My department has asked me and my colleagues to examine the teacher certification tests in Michigan, and while my colleagues took the much more challenging Professional Readiness Exam (PRE) that all would-be teachers must pass, I took the subject area test.

Last night, I got the results from the test. I am pleased to say that I passed. But what surprised me was the paucity of data Pearson actually sent me. Here is the report in its entirety:

Screen Shot 2015-02-14 at 4.06.55 PM

As the image shows, I scored ++++ in all four areas of the test. A key included in the report indicates that ++++ means I answered “most of the questions correctly.” It is the highest of four possible ratings. But that is it. No comparison to other test takers on that day. No specific data on which items, if any, I missed on the test. Full disclosure: I know I missed one question, but I was ready to fight against what I thought was a poorly worded question. But the report did not let me see this information.

This is probably par for the course for all major standardized tests, but the truth is, Pearson has all of the data. If I wanted to compare my test results against all other English majors taking the test in 2014, I should have been able to do so, even on subareas, and perhaps down to the individual question type, as in, “56 percent of all other English subject area test takers also got question 32 (parallel structure) incorrect.”

That at least would tell me, hypothetically, which specific kinds of questions I could study for future retakes, if I had failed. Does Pearson have this much data? Absolutely: they have reams on it on individual questions–this particular subject area test has been unchanged for at least the last five years.

On a related note,GVSU even has a subscription to Mometrix, a database dedicated to offering study tips for teacher certification tests.

Download PDF
Posted in News | Tagged | Comments Off on Pearson and Data: It’s a One-Way Street

Google Classroom: Bare-Bones Course Management System

My opposition to Blackboard has been pretty well established over the years that I’ve kept this blog. I have advocated for free and open course management tools such as WordPress, Wikispaces, Ning (when it was good), Mightybell, Weebly, and other sites and services. These past two week, however, I have been taking my first Blackboard seminar at the university, with the explicit purpose of getting certified to teach online and hybrid courses. I admit I have not been inside Blackboard in a long, long time, and it seems like the number of tools it offers instructors has expanded, and expanded, and expanded. Suffice to say that if can do it on the web, you can probably do it in a less cool, more complicated way inside of Blackboard. This means recording audio, uploading large video files, chatting with users in real-time, and organizing content. It’s not all bad.

Still, there the high cost of the institutional license. And the closed nature of the system, with no contact from the real web. And the way Blackboard institutionalizes classroom discussions into exercises in teacher-pleasing. I don’t mind creating a course inside of Blackboard (either hybrid on entirely online), but I’m not sure it would ever be my first choice.

One interesting new player in the learning management system market is Google Classroom, which is available (free) for users of Google Education Apps, the suite of Google tools that includes Google Drive. GVSU uses Google Apps, so I have access to Google Classroom. I’ve been playing around with it for the last few days.

One of the chief faults of Blackboard is its unnecessary complexity, but Google Classroom is dead simple. Maybe too simple. Really, there is only one main interface/page, where students view the “stream” that has been posted by the administrator, and maybe other students, but I haven’t figured this out yet. Posts can include video, attached documents, or Google Drive items such as forms or spreadsheets. Classroom interfaces with the Google Drive of the administrator, who can decide whether to let students edit or just view the document. That’s about it–no gradebook, dedicated discussion board, or anything. Anyone else out there using it?

Download PDF
Posted in Cloud Computing, News | Tagged , | Comments Off on Google Classroom: Bare-Bones Course Management System