About Nesta

Nesta is an innovation foundation. For us, innovation means turning bold ideas into reality and changing lives for the better. We use our expertise, skills and funding in areas where there are big challenges facing society.

Burning the transcript?

Online content creation, how the internet is generating a new kind of qualification, and some implications for education

Proving it

It’s partly about signalling ability. Even if university studies are not directly useful for work, there are still economic benefits from having a degree. This is because employers can’t readily know job applicants’ ability, and don’t entirely trust their self-assessments. It’s therefore useful to have a way of demonstrating potential to employers, such as marks in independent tests like degrees.[1]

The internet is however changing the need to rely on academic qualifications as an indicator of ability. Online platforms like the software collaboration platform GitHub, video sharing websites like YouTube and question and answer sites such as the programming site Stack Overflow provide ways to display skills to employers. The development of social networks also makes it easier to distribute content and build reputation.

Demonstrating ability this way is arguably more realistic and meritocratic than university qualifications. The content people display can be directly relevant for work, and by contrast to universities, where there are equality of access questions, you don’t have to go through the hoops of the formal system. All that’s needed are a computer with an internet connection, access to software (which is increasingly available in open source form).[2] Although it is also true that people need to come across the activity without the guidance of an academic institution.

People, admittedly, don’t typically work up profiles on GitHub and YouTube with the end goal of demonstrating capabilities in quite the same way they undertake a course of study for qualifications. The motivations for generating online content are varied. It could be that the activity is valued in itself and there may (or may not) be a direct ambition of gaining respect and ultimately money as a result.

With an increasingly educated and globalised workforce there is though likely to be more pressure to stand out from the competition. One way people could respond is through the greater use of online portfolios of work (It is arguable that something similar has already happened with the rise of internships and master’s degrees). It may be that online content is not yet recognised by employers, so there is less economic incentive to generate it, but that could change and in some areas already is, facilitated by the scale of some of these online content sharing platforms and the use of crowd sourcing to rate content.

Rated

A practical issue with online user-generated content online is that it can be very diverse, meaning it is difficult or time consuming to compare a large number of candidates. It is therefore easier to look at some high level measures of the content quality (i.e. something like a mark) rather than reviewing it directly.[3]  Analogously, employers want to know applicants’ grades, they don’t typically want to read the exam papers.

Mechanisms to rate contributions to online content exist. Stack Overflow has for example a points based system where users’ questions and answers can be upvoted and downvoted by other site users. This allows users to win points and badges that recognise their status on the platform, and ultimately rewards them with privileges on the site. There are also ways content can be quantified on GitHub such as the numbers of projects initiated, how often a person’s code has been reused by others (“forked”) and the number of collaborators on projects initiated by the person (see this post on sourcing job candidates on GitHub for more discussion).

In the case of Q&A sites, while the rating is decentralised, it is arguable that the platforms’ scale is central to their effectiveness (Stack Overflow has 3.5 million users[4]).People are more likely to pose questions on a site where there are lots of people to answer them. Conversely, if there aren’t many people posing questions there are less likely to be questions people feel able to answer, and so it is harder to start to build reputation on a site. There also won’t be a large bank of answered questions to drive people to the site in the first place.[5] The size of the platform also gives the ratings visibility and credibility with employers.

There is a sense in which Stack Overflow is acting as a massive educational institution where large numbers of people learn from each other, set each other mini exams, and over time get a form of public certification in return. People are already submitting their Stack Overflow profile/ratings and GitHub profile when they apply for jobs, although there is also debate as to how useful this is (see for example this discussion about Stack Overflow and this discussion about GitHub).[6]

Programming apart, it remains to be seen how well these kinds of models extend to other subjects. Stack Exchange (the wider platform Stack Overflow sits within) operates similar sites for other subjects, but as yet none have reached the scale of Stack Overflow, and perhaps programming is particularly well suited to this format.[7]  It does though seem plausible that over time both the sophistication with which online content is rated and the range of subjects/contexts that this is found to work for could increase.

Copying best practice

This is not to say that online content and its crowd sourced rating will supplant formal qualifications. There are advantages to learning things face to face, and online learning may not be a method that works for all. University curricula can be quite standardised, and are therefore more of a known quantity for employers than a standardised rating of varied online content (although platforms can, and do, discuss these bodies of knowledge, with the sites in this context potentially replacing established text books). Learning material that is not directly applicable to a work context can be a good thing, purely because it opens up new ways of thinking or is enjoyable in itself. Creating online content relies on having the time to do this, and while financing models exist to fund university studies, the same is not yet true for creating online content, except in so far as it is covered by existing business finance channels, or potentially through emerging crowdfunding platforms. What is learnt or the qualifications earned aside, university is just as much about the social experience.

There is though a case for greater recognition of the implications of online content within education. One of the issues with Q&A sites and other online sources is that they are sometimes seen as undermining traditional educational channels, with students copying answers off online sources.  However, building on the work of others is a reality of work and it is arguable that education should accept this and find ways to use assessment to recognise it, rather than meeting traditional expectations around plagiarism, although the extent to which this is true may differ by subject. There is for example a difference between using an existing software library to solve a problem and taking verbatim a whole section of someone’s writing and reusing it in an essay.

Another implication may be that universities should encourage students to build up an online portfolio of work. One reason for this is as something they can use to attract employers. Aside from being a route to securing a job, it can also be seen as a means to encourage creativity and, ultimately, entrepreneurship. Directly producing new content as part of a degree is a better measure of creativity than exams, and ultimately it’s being able to ask and answer new questions that leads to innovation. Technology may not burn the transcript, but it should ignite some changes in it.

We would be grateful for thoughts and examples of where online content creation is being used in new ways in education. The subjects and contexts where this approach works best and where it doesn’t and how issues relating to reuse of content are best dealt with? Your comments are very welcome below




[1] This argument was formalised by the Nobel prize winning economist Michael Spence who won the prize for his work on analysing markets with asymmetric information. Spence developed a model where he showed that even if educational qualifications had no effect on productivity in work their signalling effects to employers still meant there were benefits for certain people in studying to obtain them.

[2] It should be acknowledged however that it has been argued that using open source projects as a means for recruitment, makes it harder for those who have less time, such as those looking after children or whose work is bound by confidentiality requirements making it harder for them to contribute in this way http://www.ashedryden.com/blog/the-ethics-of-unpaid-labor-and-the-oss-community

[3] One implicit measure of content quality that is frequently used is position in search engine rankings, however this is not to the authors’ knowledge, as yet, based on a direct assessment of content quality, as distinct from inferring relevance from other measures such as the centrality of a page in the network of the internet, the extent to which it contains relevant search terms, other things the searcher has looked for etc. etc.

[4] Sourced from Stack Overflow October 2014.

[5] For a discussion of the business model of Stack Overflow see M. Sewak et al. “Finding a Growth Business Model at Stack Overflow, Inc.” Stanford CasePublisher 204-2010-1. 18 May 2010.

http://web.stanford.edu/class/ee204/Publications/Finding%20a%20Growth%20Business%20Model%20at%20Stack%20Overflow.pdf

[6] Stack Overflow also has a jobs site allowing companies to advertise jobs to the large number of people using the site, and users to post their profiles for potential employers to see.

[7] It is a technical subject so knowledge is very transferable i.e. there are common questions and answers, and the large number of people trying to solve similar questions with similar tools around the world creates a liquid market for the exchange of knowledge.  The modular character of programming means that projects are typically split into smaller sub-projects which may solve problems that are common across quite different contexts. Many people are self-taught so are used to finding answers for themselves and contributing back to the communities that helped them along the way. It is also arguable that there are some technical aspects to this, such as the practice being easily described through forums (as opposed to say something like dance), but it’s also cultural and based on a precedent that this what is people do. There is also a safety aspect in that users feeling their way through a field in a self-directed way via forum posts does not quite have the potential downside implications it could have in other contexts such as surgery or rock climbing.

 

Author

John Davies

John Davies

John Davies

Principal Data Scientist, Data Analytics Practice

John was a data scientist focusing on the digital and creative economy. He was interested in the interface of economics, digital technology and data.

View profile
Oliver Quinlan

Oliver Quinlan

Oliver Quinlan

Head of Impact and Research, Raspberry Pi Foundation

Oliver was a programme manager for Nesta’s digital education projects. He is now Head of Impact and Research at the Raspberry Pi Foundation.

View profile