Hi – I’m Norman Bier, from Carnegie Mellon’s Open Learning Initiative.
There’s been plenty said about the past and the present of Open Education –
so now let’s talk about the future; .
Technology has been one of the major drivers of OER – replicability w/o cost
continues to be a necessary condition for OER to succeed,
and many are looking to ongoing technological affordances to provide
other ways mechanisms for transforming education.
Most frequently I hear suggestions that technology
can be transformative by letting studns learn anywhere and any time,
or that tech can provide rich simulations and labs to students;
others, especially in the OER space, continue
to believe the access that technology enables will be transformational.
While these affordances are definitely important,
at OLI we’ve felt that the technology’s greatest potential is in the ability
to collect and apply data – web-based courseware gives us the ability
to embed assessment into every student learning opportunity,
and to leverage the data capture from these interactions to fundamentally
improve learning – driving better hints and feedback to learners;
offering faculty more focuses queues on where their students are struggling
and where they’re succeeding.
More broadly, we can interpret these interactions in ways that can
continuously improve the design of our courses – and in the aggregate,
this information can feed back into the research community,
advancing our understanding of how humans learn.
Collecting and using this learning data is controversial –especially given some
of the recent data abuses and breaches – and I’m hearing some leaders
in our community calling for strong solutions,
ranging from dramatically locking down data to renouncing the collection
of data entirely.
That’s an important conversation, but it’s not the one that I’m engaging
in right now – I think we are already in a world where data-driven approaches
and materials are already being developed,
adopted and embraced – by vendors, by schools,
by foundations and by government.
The future is already here.
And to be clear – these approaches have already demonstrated an amazing
potential improve outcomes, lower costs and enable new approaches
and pedagogies.
Optimists among us are right to look at these approaches as an opportunity
to move the needle in education, especially among vulnerable learning
populations.
But the pessimists among also have real concerns,
in that this seems to provide publishers with a sudden opportunity
to reconsider business models as offering subscription services just
at the time when real progress is being made on opening content.
My concerns are a little more direct – effective use of data drive approaches
rely on instrumented content, that is, content that can provide meaningful
assessments of student learning.
It relies on systems that can capture and store
that data, on algorithms that can analyze and make
sense of that learner data.
And it relies on displays and dashboards that
can make that analysis actionable and understandable.
Vendors are out ahead in building out these systems and the courseware
that drives them, and open education is getting left behind in this race,
with a continuing emphasis on developing static open content,
and without systems to leverage what instrumented content is open.
So even as we see a broader recognition and use of OER,
we also see publishers moving upmarket into data-driven systems,
leaving us with a real risk of open ending up w/ a 2nd class solutions
and OER consumers –especially galling when you realize that many of these
vendors are populating their systems with OER.
When vendors are the primary drivers of this approach,
it also leaves us with a reach danger of vendor lock-in – and all of that data
and content ends up stuck in proprietary systems if you leave … or if they go
out of business.
But I’m even more concerned that in this model,
the algorithms and models become secret, black box systems.
With no visibility into the ways that data is being analyze or the ways that
predictions and recommendations are being made,
we and are learners become increasingly dependent upon systems that we don’t
understand, that that actually might not be making particularly good suggestions
or recommendations.
Beyond the lack of direct visibility into these systems –that is,
the ability to look into the code, there’s also remarkably little
secondary visibility; research on the efficacy of these systems is few
and far between.
Too often what passes as evidence is actually marketing materials
masquerading as research, and we’ve seen a few examples of what appeared to be
rigorous research into analytics systems turn out to be…well…just wrong.
Absent transparency into the systems that power some of the most popular
analytics systems, some educators have looked at the inputs and outputs
of the system to try to reverse-engineer what claim to be sophisticated,
statistical models, only to find that in some cases some simple averaging
approaches can replicate the vendor’s results.
By itself, this should all be enough to make the case for the Open Education
movement to set its sights on establishing and using
fully open analytic systems, powered by fully open models and algorithms.
At a basic level, these kinds of open systems
can guarantee that Open Education can continue
to compete with (and avoid being exploited by)
closed, proprietary approaches.
Equally important, such systems not only avoid the data lock-in problem,
by establishing clear standards for data and portability, such systems
are more likely to force vendors to provide more options for porting data
into and out of their systems.
Most important, though, an open approach to these types of system
provide full transparency into the workings of the algorithms and models
that are driving recommendation.
This is absolutely essential to understanding these system,
and essential to making informed decisions on how
you leverage and interpret the recommendations these systems make.
Equally important to me, though, is that this type of transparency is essential
for any type of scientific exploration of the algorithms and models –
understanding how (or if) they work to begin with.
Engaging in the difficult work of refining them
and making them more useful.
And perhaps most importantly, investigating the assumptions
that are implicit in the algorithms and models.
We’ve seen time and time again that the implicit biases of developers find
their way into algorithms – from the algorithms that power social media
to the software the drives automatic hand dryers,
we’ve seen these biases at work.
To borrow from Audrey Watters, our algorithms are not neutral – I can make
every effort to identify and compensate for my biases as I develop my own
algorithms, but like the bulk of an iceberg,
by definition the bulk of our implicit biases are hidden from our own view.
When I develop a model to to identify and predict learning,
it’s pretty likely to prioritize and privilege patterns and behaviors that
look like learning to me – and to deemphasize motivations and actions
that don’t’ align quite as well my experience.
As proud as I am of the work and experiences that have lead me
to Carnegie Mellon, I’m under no illusions that these necessariliy
the types of common experiences and motivations upon which we should be
basing our common understanding and models of learning.
If none of us can fully identify and avoid their own biases,
then we must have a systems that allow for broader exploration, identification
and remediation of these biases.
And the transparency that’s inherent in the open approach is the best way that
I know to ensure this work can happen.
So, I believe that the next battle for the Open Education movement is
in the data-driven education space, and I think that the creation of open
systems, algorithms and models --- along with the instrumented OER that can
drive it – is absolutely critical.
I think there’s some real urgency in making progress on this dimension,
and I’m worried that too much of our attention continues to be
on the development of static, open textbooks.
And while we’ve focused on other things, proprietary solutions have made
a enormous inroads in claiming the data-driven space for their own.
A quick note for those of you watching this as part of the EdX Open Education
MOOC – there’s been some great research that suggests that watching videos like
this one are one of the worst ways for you to learn – it’s much more effective
for you to engage in richer, learn-by-doing activities.
Hopefully Wiley and Siemens team have some great have some great activities
lined up, but in case they don’t: I have some home work for you:
Take a quick look around google and identify
some of the leading options for learning analytics
and data-driven, instrumented courseware.
What can you say about the efficacy of the solution they offer?
Do you have any clarity on how their algorithms operate?
Now, see what you can find on current work in open analytics and open
learning models – who’s doing the most interesting work?
Are there systems that are currently adoptable?
If not, why not – what needs to happen before there are practical,
open solutions in place?
If you have a moment, take a look at some of the online conversations
happening in response to the new Creative Commons Open Education
Platform; Creative Common’s Cable Greene has been outspoken in promoting
the need for open systems in this space; do you see this reflected
in the platform’s current conversations?
Finally, take a few minutes to read Audrey Watters’ essay “Identity,
Power and Education’s Algorthim’s” – and then make a list of the kind of biases
you can imagine creeping into the kinds of algorithms we’re exploring.
Which ones are the most concerning?
How can an open approach help mitigate those concerns?
Hope you’ve found this interesting – questions?
Comments?
Howls of disbelief & derision? let me know on twitter: @normanbier